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ABSTRACT 


This  report  discusses  continuations  of  the  work  reported  in  Special 
Scientific  Report  No.  1  on  this  same  contract.  Part  1  (Threshold  Logic) 
consists  of  a  comprehensive  survey  of  threshold  logic,  a  geometric  result 
relevant  to  estimating  the  number  of  threshold  functions,  generalizations  and 
strengthening  of  known  bounds  on  the  logical  capabilities  of  threshold  gate 
networks,  computer-aided  work  on  the  realization  of  arbitrary  functions  by 
networks  of  three-input  majority  gates  (including  a  list  of  realizations  for 
every  type  of  four-argument  switching  function),  and  a  comparison  of  two 
methods  for  synthesis  of  very  large  threshold  gates:  the  well-known  Bayesian 
approach  and  a  geometric  alternative.  The  latter  method  is  shown  to  be 
preferable. 

Part  2  (Reliability  of  Switching  Networks)  presents  a  survey  of  several 
important  schemes  for  introducing  redundancy  into  a  combinational  network 
for  the  improvement  of  reliability  —  comparisons  are  made  with  the  recursive 
triangle  system  (see  Special  Scientific  Report  No.  1) ;  some  extensions  of 
the  previous  analyses  of  recursive  triangles  (specifically,  more  general 
results  on  the  "and"  function,  study  of  a  nonsymnetric  function,  ABvCD,  and 
generalization  to  the  "nor"  function) ;  and  initial  results  on  the  incorpora¬ 
tion  of  memory  and  feedback  to  allow  the  use  of  fewer  basic  gates  in  a  time- 
shared  fashion. 
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The  two  major  parts  of  this  report  cover  the  two  branches  of  our  in¬ 
vestigation  of  the  theory  of  adjustable  switching  networks;  Threshold 
Logic  and  Reliability  of  Switching  Networks.  Each  part  thoroughly 
sunmarizes  and  analyzes  the  research  performed  during  the  last  year. 
Research  performed  in  the  first  year  of  the  contract  has  been  reported 
in  Special  Scientific  Report  No.  1;  the  present  report  should  be  considered 
a  continuation  of  the  latter,  where  definitions,  motivations,  his¬ 
torical  background,  etc.,  can  be  found.  Work  on  the  theory  of  adjustable 
switching  networks  is  continuing. 

A  cumulative  list  of  the  Scientific  Reports  issued  on  this  contract 
follows: 


Scientific  Report  No.  1.  Winder,  R.  0.,  "More  About  Threshold  Logic", 

AFCRL  702,  July  14,  1961. 

Scientific  Report  No.  2.  Brzozowski,  J.  A.,  "Reliability  of  Triangular 

Switching  Networks  with  Intermittent  Failures", 
AFCRL  785,  August  14,  1961. 

Scientific  Report  No.  3.  Levy,  S.  Y.,  "Triangular  Rectifier  Networks", 

AFCRL  786,  August  23,  1961. 


Scientific  Report  No.  4. 


Miiller,  H.  S.,  Winder,  R.  0.,  "Majority  Logic 
by  Geometric  Methods",  AFCRL  792,  July  13,  1961. 


Scientific  Report  No.  5. 
Special 

Scientific  Report  No.  1. 


Amarel,  S.,  Cooke,  G.,  Winder,  R.  0.,  "Majority 
Gate  Networks",  AFCRL  793,  August  14,  1961. 

Amarel,  S.,  Levy,  S.  Y.,  Winder,  R.  0.,  "Theory 
of  Adjustable  Switching  Networks",  AFCRL-62-318, 
April  30,  1962. 


Scientific  Report  No.  6. 


Winder,  R.  0.,  "Threshold  Logic  in  Artificial 
Intelligence",  AFCRL-63-6,  November  15,  1962. 
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I.  INTRODUCTION 


Part  1  of  Special  Scientific  Report  No.  2  consists  of  several  sections: 
Section  II  and  Section  VII  (References,  pg. 39  )  constitute  a  comprehensive 
survey  of  threshold  logic.  Sections  III,  IV,  V,  and  VI  describe  the  present 
state  of  several  lines  of  investigation  under  this  contract.  (Whereas  at  the 
end  of  the  first  year  of  this  contract  several  primary  lines  of  investigation 
had  terminated,  so  that  a  final-type  report  —  Special  Scientific  Report  No.  1, 
Part  1  —  was  written,  this  is  not  the  case  this  year.)  The  Appendix  of  Part  1 
(Section  VIII)  is  a  listing  of  networks  which  resulted  from  the  investigations 
reported  in  Section  V. 

Threshold  logic,  especially  as  reported  in  Special  Scientific  Report 
No.  1,  has  been  primarily  motivated  by  an  interest  in  computer  applications 

—  design  with  threshold  gates.  Recently,  the  interesting  possibilities  of  a 
second  major  field  of  application  have  become  apparent:  To  define  complicated 
decision  functions  of  many  variables,  students  of  artificial  intelligence 
almost  invariably  have  used  the  simple  and  natural  idea  of  "linear  separation" 

—  their  basic  functions  have  turned  out  to  be  threshold  functions.  It  seems 
that  both  biological  and  engineering  reasons  lie  behind  this:  First,  the 
psycho-physiological  theories  of  learning  and  perception  (e.g.,  Hebb)  have 
employed  theories  of  neural  nets  (McCulloch  and  Pitts);  these  neural  models 
are,  in  fact,  typical  threshold  gates.  Second,  investigators  of  self-adjust¬ 
ing  switching  devices  have  found  neuron-like  elements  to  be  easily  controllable. 

Threshold  logic  has  much  to  offer  students  of  artificial  intelligence. 
Furthermore,  an  increased  interaction  will  surely  produce  interesting  new 
developments  in  threshold  logic.  For  example,  in  Section  VI  we  consider  an 
important  common  area  between  the  fields:  the  synthesis  of  a  single  threshold 
gate  for  purposes  of  pattern  recognition.  The  frequently  used  probabilistic 
approach  to  the  problem  is  outlined,  its  main  shortcomings  discussed,  and  an 
alternative  given,  which  arises  from  the  switching  theoretic  point  of  view. 

It  is  shown  that  the  probabilistic  results  are  always  worse  than  the  crudest 
form  of  the  switching  theoretic  results. 
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Section  II  provides  a  general  survey  of  the  relevant  sections  of  thres¬ 
hold  logic.  A  guide  to  the  literature  is  provided,  specifically  slanted  toward 
the  student  of  artificial  intelligence.  This  is  an  attempt  to  stimulate  the 
Interest  of  such  people  in  threshold  logic,  to  make  the  literature  easily 
accessible,  and  to  encourage  their  continued  attention. 

The  problem  of  estimating  the  number  of  threshold  functions,  out  of  all 
switching  functions  of  a  given  number  of  arguments,  is  an  important  one.  It 
can  be  shown  equivalent  (see  Special  Scientific  Report  No.  1)  to  the  geometric 
problem  of  partitioning  Euclidean  n-space  by  a  certain  collection  of  hyper¬ 
planes.  In  connection  with  this  approach.  Section  III  reports  an  important 
theoretical  step  toward  a  resolution  of  the  problem.  Work  on  application  of 
the  result  continues. 


In  Section  IV  a  generalization  of  earlier  estimates  (or  more  specifically, 
upper  bounds)  is  made. 

of  incompletely  specified  threshold  functions: 


This  generalization  gives  upper  bounds  on  the  number 

,n 


Suppose  m  of  the  possible  2 
input  combinations  to  a  gate  have  given  specified  outputs  associated  with  them. 
Out  of  the  entirety  of  2m  such  incompletely  specified  functions,  averaging 
over  the  ^2nJ  choices  of  the  ra  points,  less  than  m11  will  be  realizable  with  a 
single  threshold  gate.  This,  for  example,  puts  severe  restrictions  on  the 
logical  capabilities  of  a  network  such  as  the  Perceptron,  where  adaptation 
is  possible  on  only  one  threshold  gate.  Section  IV  also  strengthens  an 
earlier  result  of  Cameron:  The  random  function  of  n  arguments  will  require 
more  than  V 2^n+^//  n  threshold  gates,  interconnected  in  a  network,  for 
realization.  A  bound  for  the  likely  size  of  networks  for  realization  of 
incompletely  specified  functions  is  also  obtained. 


Section  V  reports  computer-aided  investigations  of  the  problem  of  realiz¬ 
ing  arbitrary  functions  with  networks  of  simple  three-input  majority  gates. 
Results  are  incomplete  and  inconclusive;  much  work  remains  to  be  done.  An 
outcome  of  the  experimentation,  given  in  the  Appendix  (Section  VIII),  is  a  set 
of  threshold  gate  network  realizations  for  each  type  of  four-argument  switching 
function  —  the  networks  have  been  proved  minimal  in  their  number  of  stages. 
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The  ideas  of  Section  VI,  mentioned  above,  have  introduced  two  new  lines 
of  investigation,  both  of  which  will  be  studied  in  the  third  year  of  this 
contract.  The  first  concerns  basic  limitations  of  the  Bayesian  approach  to 
threshold  gate  design.  The  limitations  stem  from  certain  basic  "independence 
assumptions",  necessary  for  the  Bayesian  synthesis  procedure.  The  first 
theorem  of  Section  VI  states  that  if  a  threshold  function  with  equiprobable 
input  combinations  satisfies  the  independence  assumptions,  it  must  be  a  trivial 
function.  A  strengthening  of  this  result  is  now  being  investigated:  Any 
fraction,  with  any  input  distribution,  that  satisfies  the  independence 
assumptions,  must  be  trivial.  In  other  words,  the  Bayesian  approach  cannot 
be  applied  to  nontrivial  functions  with  any  guarantee  of  success.  Important 
consequences  for  artificial  intelligence,  and  specifically  for  adaptive 
systems  of  threshold  gates,  are  under  study.  The  second  new  line  of  inves¬ 
tigation  concerns  the  switching  theoretic  countersuggestion  to  the  Bayesian 
synthesis  procedure,  outlined  in  Section  VI;  it  will  also  receive  more 
attention.  The  development  of  heuristic  programs  for  the  synthesis  of 
threshold  gate  networks  is  also  contemplated. 
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II.  SURVEY  OF  THRESHOLD  LOGIC 
by  R.  0.  Winder 

Threshold  logic  areas  of  interest  to  investigators  of  artificial  intelli¬ 
gence  are  outlined,  and  some  problems  whose  solutions  would  be  significant  in 
artificial  intelligence  are  suggested  in  this  section. 

Host  of  the  threshold  logic  literature  deals  with  one  or  more  of  the  follow¬ 
ing  three  areas:  Conditions  which  functions  must  satisfy  to  be  threshold  functions, 
algorithms  for  determining  the  existence  and  nature  of  realizations,  and  methods 
(heuristic)  for  synthesizing  networks  built  from  threshold  gates,  often,  in  partic¬ 
ular,  from  three-input  majority  gates  (AB  +  AC  +  BC).  In  the  following  we 
discuss  the  large  body  of  simple  transformations  and  other  properties  known,  then 
the  necessary  conditions,  one-element  test-synthesis,  network  synthesis,  and 
finally  the  more  important  miscellaneous  results,  (see  [Winder-3]  for  an  earlier 
survey). 

A.  SIMPLE  PROPERTIES 

The  following  are  typical  simple  facts  about  threshold  functions: 

1.  Realizing  weights  and  threshold  are  not  unique. 

2  All  threshold  functions  can  be  realized  with  integral  weights  and 
threshold. 

3.  Any  pair  of  (unequal)  numbers  (i.e.,  0  and  1,  -1  and  +1,  etc.)  can 
be  used  as  the  numerical  equivalents  of  the  switching  variables  in 
order  to  define  threshold  functions.  (The  same  weights  can  be  used 

in  different  systems;  the  appropriate  thresholds  are  easily  calculated.) 

4.  Given  a  threshold  function  f,  functions  derived  from  f  by  permuting  or 
complementing  arguments,  or  by  dualizing  or  complementing  f,  or  any 
combination  of  these,  are  also  threshold  functions.  (Again,  appro¬ 
priate  transformation  rules  are  easy  to  define.) 

A  general  treatment  of  these,  and  most  of  the  topics  mentioned  below,  can 
be  found  in  the  papers  [ElgotJ,  [Gabelman-1],  [Muroga-2,  5],  and  [Winder-1, 

4].  These  are  relatively  comprehensive  papers,  with  much  duplication  of  the 
basic  material.  [Coates-Lewis]  "Iso  covers  a  wide  range  of  material,  but  with 
a  specific  slant  toward  the  test-synthesis  problem. 
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B.  NECESSARY  CONDITIONS 


The  simplest  condition  which  all  threshold  functions  must  meet  is 
unateness*  discussed  originally  in  [Muroga-1],  [McNaughton] ,  and  [Paull- 
McCluskey].  [Paull-McCluskey] ,  [Winder-1],  and  [Muroga-2]  generalize  this 
condition,  obtaining  an  infinite  family  of  necessary  conditions:  1-mono- 
tonicitv  (equivalent  to  unateness),  2-monotoniclty .  etc.  A  function  satis¬ 
fying  all  of  these  conditions  is  completely  monotonic.  [Winder-1,  4]  give 
the  most  complete  treatment  of  these  ideas,  including  an  important  function 
due  to  E.  F.  Moore  of  Bell  Laboratories  --  a  twelve-argument  function  which 
is  completely  monotonic,  but  is  not  a  threshold  function. 

Because  complete  monotonicity  fails  to  characterize  threshold  functions, 
this  idea  in  turn  is  generalized  in  various  ways  in  [Elgot]  and  [Gabelman-1] . 
[Winder-2,  4]  discusses  these  generalizations  further,  settling  some  questions 
left  unanswered  in  [Elgot]  and  [Elgot -Muroga ] ,  and  concentrating  on  a  second 
infinite  family  of  necessary  conditions:  2-assummabilitv  (equivalent  to 
complete  monotonicity),  3-assummabilitv.  etc.  A  function  satisfying  all  of 
these  conditions  is  proved  in  [Elgot]  to  be  a  threshold  function.  This 
necessary  and  sufficient  condition  was  published  about  the  same  time  in 
[Chow-1].  It  is  discussed  in  terms  of  convex  sets  in  [Highleyman]  and  [Gabelman-2 ] . 

These  various  necessary  conditions  have  obvious  importance  in  artificial 
intelligence.  Even  for  sparsely  specified  functions  (where  most  inputs  do  not 
have  outputs  specified)  of  many  arguments,  functions  can  often  be  shown  to  he 
nonthreshold  functions  by  simple  observations.  More  work  is  needed,  however, 
to  adapt  the  ideas  more  specifically  to  the  case  of  sparse  specification,  many 
inputs. 


*A  function  is  unate  when  it  can  be  expressed  by  a  Boolean  expression  in 
which  each  variable  appears  uniformly:  everywhere  with  negation  or  every¬ 
where  without. 
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C.  TEST-SYNTHESIS 


The  question:  "Is  a  given  function  a  threshold  function,and  if  it  is,  what 
weight  and  theshold  assignments  realize  it?"  is  easily  seen  to  be  equivalent  to 
the  consistency  and  solution  of  a  certain  system  of  linear  inequalities  I,  on 
the  weights  and  threshold  as  "unknowns"  --  [McNaughton] .  If  the  function  has  n 
arguments,  I  contains  2n  inequalities.  The  theory  of  k-monotonicities  can  be 
used  to  reduce  this  system  to  manageable  proportions  —  in  various  forms,  most 
of  the  papers  to  be  mentioned  in  this  section  make  use  of  ideas  equivalent  to 
at  least  1-monotonicity,  and  often  2-monotonicity.  The  problem  of  the  equivalence 
of  these  reduced  systems  I*  with  the  original  is  discussed  in  [Winder-1,  4]. 

Many  different  methods  of  solution  for  I'  have  been  proposed.  Algebraic  solution, 
which  involves  successive  elimination  of  unknowns  and  yields  a  specification  of 
all  solutions,  is  discussed  in  [Elgot]  and  [Winder-1,  4]. 

The  important  idea  of  solving  I '  by  linear  programming  is  discussed  in 
[Minnick],  [Stram-1],  [Muroga-1],  and  [Einhorn].  Integers  are  usually  obtained 
under  a  condition  that  their  sum  (absolute  values)  be  minimum;  by  a  technical 
device.  Integer  programming  has  been  avoidable  except  for  some  specially  con¬ 
structed  examples  of  E.  F.  Moore  and  [Winder-2,  4]  (thus  settling  the  other  open 
problem  of  [Elgot-Muroga]).  A  game  theoretic  approach  to  the  solution  of  I'  is 
discussed  in  [Akers-1].  Both  the  linear  programing  and  game  theoretic  approaches 
produce  just  one  realization,  or  else  prove  that  there  are  none. 

The  procedure  of  [Coates-Lewis]  and  [Coates-Kirchner-Lewis ]  is  a  specialized 
algorithm  which  algebraically  produces  a  single  solution,  or  proves  that  there  are 
none.  A  geometric,  heuristic  procedure  is  described  in  [Stram-1,  2].  In 
[Dadda-1,  2]  methods  suitable  for  small  n  are  given.  In  [Varshavskii-1 ]  a 
fallacious  procedure  is  summarized. 

The  question  of  obtaining  minimal  integral  realizations  (the  sum  of  absolute 
values  of  weights  and  threshold  minimized)  is  very  nicely  solved  in  [Gabelman-1, 

3  ;  the  procedure  is  refined  and  rigorously  established  in  [Winder-4].  Synthesis 
when  the  function  is  incompletely  specified  is  treated  in  [Winder-1,  4]  (alge¬ 
braically,  with  not  too  many  inputs  unspecified)  and  in  [Singleton]  (by  matrices, 
with  not  too  many  inputs  specified).  The  intermediate  problem  has  obvious  import¬ 
ance  in  artificial  intelligence  and  in  character  recognition;  the  ideas  described 
in  [Winder-6]  (and  section  VI  of  this  report)  may  provide  a  start  in  this  direction. 
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All  methods  discussed  above  (except  those  of  [Winder-6])  are  switching- 
theory  oriented,  in  the  sense  that  they  work  exactly,  and  are  not  feasible  when 
the  number  of  arguments  becomes  large  (over  20,  say).  It  is  likely  that  arti¬ 
ficial  intelligence  motivations  will  soon  be  producing  some  useful  results  for 
large  n  (and  for  largely  unspecified  functions). 

D.  NETWORKS 

The  question  of  synthesizing  a  given  function  in  a  network  of  threshold 
gates  is  a  typically  difficult  switching  theoretic  problem.  (A  single  threshold 
gate  realizes  a  threshold  function;  a  network  of  threshold  gates  corresponds  to 
a  composition  of  threshold  functions.)  It  has  been  considered  under  various 
constraints:  In  computer  design  applications,  tolerance  considerations  dras¬ 
tically  limit  the  number  of  inputs  that  can  be  allowed.  [Muroga-4]  is  a  general 
discussion  of  this;  [Muroga-5]  goes  into  important  details.  There  are  several 
papers  on  synthesis  by  networks  of  three-input  majority  gates  (AB  +  AC  +  BC) , 
the  simplest  threshold  functions  beyond  the  conventional  NOT,  OR,  AND,  and  their 
variants.  The  methods  can  be  divided  according  to  the  means  of  function  repre¬ 
sentation:  Working  algebraically,  [Lindaman-1,2],  [Cohn-Lindaman],  and 
[Akers-2]  discuss  various  transformations,  analogous  to  the  ordinary  Boolean 
transformations,  which  may  be  helpful  in  synthesis.  [Winder -4]  gives  an  axiomatic 
treatment  which  may  be  helpful  in  synthesis.  [Winder -4]  gives  an  axiomatic 
treatment  which  may  be  more  useful  in  this  context  than  that  of  [Cohn-Lindaman], 

A  geometric  approach  is  described  in  [Miiller-Winder],  and  results  on  all  four- 
argument  functions  are  reported  in  [Winder-5]  (and  in  Section  VI  of  this  report). 
(Akers -3]  employs  truth  tables  in  interesting  fashion. 

The  synthesis  of  a  specific  class  of  functions  --  symmetric  functions  --  has 
been  treated  in  [Muroga-2],  [Minnick],  and  [Kautz], 

A  geometric  approach  to  the  unrestricted  problem  is  described  in  [Winder -2, 
4],  where  the  question  of  determining  whether  a  given  function  can  be  realized 
using  just  two  threshold  gates  is  answered.  An  algebraic  method,  where  con¬ 
straints  on  the  magnitude  of  the  threshold  can  be  made  conveniently,  is  given  in 
[Lewis -Coates].  [Varshavskii-2]  sunnarlzes  an  interesting  construction,  and  gives 
a  startling  (but  false,  see  below)  "theorem"  that  a  function  of  n  arguments  can 
always  be  realized  in  a  network  of  n  +  1  threshold  gates.  Many  of  the  one-element 
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test-synthesis  procedures  discussed  above  go  on,  when  a  function  ia  shown  to  be 
nonthreshold,  to  produce  a  network  in  heuristic  fashion.  Typically,  several 
fragmentary  functions  are  OR'ed  together.  A  heuristic  modification  of  linear 
programming  is  used  in  [Minnick]  to  produce  a  table  of  network  realizations  for 
all  (classes  of)  4-argument  functions. 

The  network  synthesis  is  just  beginning,  and  so  far  is  restrained  to  very 
small  n  (less  than  10).  Perhaps  the  most  promising  approach  for  the  (heuristic) 
design  of  large-n  two-level  networks  (several  "upper -level"  gates  feeding  a 
single  "output  gate",  as  in  the  Perceptron,  etc.)  lies  in  the  methods  of 
[Winder-2,  4],  where  failures  of  the  k -mono tonicity  tests  are  used  to  specify 
appropriate  upper-level  gates  --  i.e.,  each  upper  gate  can  make  the  final  decision 
as  a  threshold  function.  (Incidentally,  the  conjecture  that  the  output  gate  can 
always  be  taken  as  a  simple  many-input  majority  gate,  still  yielding  optimal 
networks,  can  be  disproved  by  counterexample.) 

E.  OTHER  MAJOR  SUBJECTS 


The  parameters  of  [Chow-2]  are  used  Implicitly  in  Section  VI.  Besides 

their  practical  use,  the  paper  cited  opens  up  a  theoretically  very  interesting 

possibility  that  threshold  functions  may  be  characterizable  by  their  parameters 

[the  m(f  )  and  m(f)]. 
i 

A  large  number  of  "functional  properties"  of  threshold  functions,  dealing 
with  various  combinations  of  functions,  chains  of  functions  obtained  by  varying 
threshold,  duality  relations,  classification  by  prime  implicants ,  and  similar 
ideas,  are  treated  in  [Muroga-2,  3,  6],  [Elgot],  and  [Gabelman-1 ] .  These  ideas 
are  rather  of  theoretical  than  practical  interest,  and  at  present  do  not  appear 
to  have  much  bearing  on  artificial  intelligence. 

R^,  the  number  of  threshold  functions  of  n  arguments,  has  been  shown  to 
be  bounded  above  ([Muroga-2]  [Winder-1  4],  [Cameron])  and  below  ([Goto- 
Takahasi],  [Muroga-2,  6])  as  follows: 


0.33n2<R  <2n2 
n 


<  <  2 


(n  >  1), 
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2 

where  2  is  the  total  number  of  switching  functions  of  n  arguments.  Thus 
threshold  functions  become  a  vanishingly  small  proportion  of  all  functions, 
as  n  Increases.  [This  suggests  that  the  often-used  idea  of  randomly  chosen 
upper-level  threshold  gates  in  a  2-level  network  is  likely  to  require  the  output 
gate  to  realize  a  nonthreshold  function.  A  rational  procedure,  such  as  that 
mentioned  in  the  section  on  networks,  is  much  more  likely  to  work  with  fewer 
upper-level  gates.  In  particular,  tentative  ideas  indicate  that  the  output 
gate,  not  the  upper-level  gates,  should  be  partially  specified  at  the  outset 
(for  example,  the  weights'  signs,  and  possibly  their  relative  absolute  magni¬ 
tudes,  might  be  determined  on  a  heuristic  or  random  basis) >  then  existing 
methods,  as  mentioned  above,  can  be  used  to  specify  appropriate  upper-level 
gates  (and  the  output  gate),  even  for  very  sparsely  specified,  many-input 
functions.  (Modifications  for  iterative  synthesis  would  be  required.)  As 
noted  earlier,  the  best  networks  are  obtained  by  allowing  the  output  gate  to 
be  as  complicated  as  is  needed.]  In  [Cameron]  it  is  shown,  using  the  upper 

bound,  that  networks  to  realize  given  n-argument  functions  will  for  most  func- 

/  1 

tions  require  exponentially  many  threshold  gates  --  something  like  2  '  . 

Bounds  on  the  size  of  integral  weights  that  will  be  required  for  some  functions 
are  found  in  [Myhill-Kautz ]  and  [Muroga-5]. 

In  Section  IV  of  this  report,  generalizations  of  these  results  are  made: 
Bounds  on  the  number  of  imcompletely  specified  functions  realizable  by  a 
single  threshold  gate  are  obtained,  and  are  applied  to  indicate  fundamental 
limitations  of  the  Perceptron-Pappa- Pandemonium  type  networks  of  threshold 

/3  1  n+* 

gates.  Cameron's  bound,  2n  ,  is  improved  to  —  2  ^  .  In  Section  III  a 
theoretical  geometric  result  is  obtained  which  may  lead  to  further  improvement 
of  the  various  bounds. 

The  classification  and  enumeration  of  classes,  for  threshold  functions, 
have  been  studied  in  [Winder-1,4]  and  [Muroga-1].  Tables  listing  representa¬ 
tives  of  all  six-argument  threshold  functions,  with  minimum  integral  reali¬ 
zations,  are  to  be  found  in  [Wlnder-4]  and  [Muroga-6],  A  more  efficient 
classification  scheme  has  recently  been  described  in  [Goto-Takahasi] :  relevant 
results  are  in  [Muroga-5]. 

Two  variants  of  threshold  logic  are  discussed  in  [Ercoli-Mercurio]  and 
[Hotz]. 
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III.  PARTITIONS  OF  N-SPACE  BY  HYPERPLANES 
by  R.  0.  Winder 

The  following  problem  has  arisen  in  the  field  of  switching  theory 
(i.e.,  computer-motivated  mathematical  logic):  Given  a  well-defined  set  of 
m  (n-1) -dimensional  linear  subspaces  in  n-dimensional  Euclidean  space,  all 
passing  through  some  given  point,  to  find  the  number  of  regions  into  which 
the  n-space  is  divided  by  the  m  "hyperplanes".  We  will  develop  below  a 
formula  for  this  quantity  which  seems  to  be  of  general  mathematical  interest. 

The  best  result  to  date  has  been  a  formula  for  the  special  case  of 
the  hyperplanes  lying  in  "general  position"  (in  a  sense  which  will  be  made 
explicit  later) .  The  hyperplanes  of  the  original  problem  failed  to  satisfy 
this  condition  --  the  formula  provided,  then,  an  upper  bound  on  the  desired 
quantity.  The  formula  is  ("B"  for  "bound") 

(";')•  a> 

See*  [Winder-1],  [Cameron],  i*° 

Definition:  A  set  of  k  (n-1) -dimensional  "hyperplanes "  in  n-di¬ 
mensional  Euclidean  space,  all  planes  passing  through  some  given 
point,  is  nondegenerate  if  the  intersection  (a  linear  subspace  of 
generally  lower  dimension)  has  dimension  n-k.  If  the  dimension  is 
of  different  parity  than  n-k,  the  set  is  odd -degenerate :  if  the 
dimension  has  the  same  parity  as  n-k,  even-degenerate. 

The  term  "general  position",  used  above,  is  defined  by  the  property 
that  every  set  of  n  or  fewer  of  the  given  m  planes  should  be  nondegenerate. 

Theorem:  The  number  of  regions  into  which  m  hyperplanes,  all 
passing  through  some  common  point,  divide  n-space  is  equal  to  the 
number  of  distinct  even-degenerate  subsets  of  the  given  m  planes 
minus  the  number  of  distinct  odd-degenerate  subsets.  (The  empty 
subset  is  included  and  is  counted  as  even-degenerate.) 

Example:  Suppose  that  the  set  of  m  planes  are  in  general  position. 
Then  the  number  of  regions  is 


*D.  T.  Perkins,  D.  G.  Willis,  and  E.  A.  Whitmore,  of  Lockheed  (Missiles  and 
Space  Division),  have  not  published  their  early  results  on  this  problem. 
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(2) 


<  '  ft)  +  ft)  *  -  -ft)  -  U)  -  (£)  '  (n+3)+  -  ±ft)  - 

because  each  subset  o£  n  or  fewer  hyperplanes,  counted  In  the  first  (n+1) 
terns,  are  nondegenerate,  and  so  even-degenerate,  and  thus  are  counted  positively 
in  the  sun.  All  subsets  of  more  than  n  hyperplanes  have  just  a  point  in  common, 
i.e.,  their  intersection  has  dimension  0.  Thus  we  have  the  alternating  signs 
for  remainder  of  the  expression.  But 


£  ft)  c-'r1*  -  o-o*  -  o. 

1-0 

Adding  corresponding  terms  in  (2)  and  (3),  we  have 

where  the  last  term  is  or  j  .  But 

ft)  - 

so  substituting  in  (4)  (except  for  if  it's  present),  we  obtain  B™ 
given  in  (1). 


(3) 


(4) 


(5) 


as 


Proof  of  theorem:  We  proceed  by  induction  on  m.  For  m»1,  there  are 

two  subsets  --  the  empty  set  and  the  given  hyperplane,  each  of  which  are  trivially 
nondegenerate,  and  so  even-degenerate.  Thus  the  theorem  would  predict  two 
regions,  which  is, of  course, correct. 

Supposing  the  theorem  verified  for  sets  of  m  or  fewer  hyperplanes,  we 

consider  a  set  of  (nri-1)  distinct  hyperplanes  (m  >  1),  H_,  H, ,  ...,  H  ,  all 

“  u  1  in 

passing  through  some  common  point.  Suppose  we  denote  the  number  of  regions  that 
they  divide  n-space  into  by 


■>n 


Hj, 


We  now  select  one  of  the  planes,  say  H^,  and  will  prove  the  following: 

11 


'•*'  Hm  j"  “  #  i  Hi>  h2>  •••>  Hm  J 

n- 1 

+  #{  HqH,,  H0H2,  ...,  H0Hm  j 


(6) 


where  HqH^  represents  the  intersection  of  the  two  planes,  and  the  right-hand 
expression  of  (6)  represents  a  subdivision  of  the  hyperplane  Hq  (an  (n-l)-di- 
mensional  Euclidean  space  in  its  own  right)  by  its  intersections  with  the  other 
planes  (since  the  hyperplanes  are  distinct,  and  since  no  pair  of  hyperplanes 
is  parallel,  the  intersections  HqIL  are,  indeed,  (n-2)-dimensional  "hyperplanes" 
in  Hq).  Although  the  hyperplanes  Hq,  Hj  ,  . ..,  are  distinct,  the  entities 

HqHj,  HqH2,  . ..,  HqH^  may  not  be;  when  we  enclose  them  by  brackets  we  are 
thinking  of  the  set,  which  may  then  have  less  than  m  members. 

Justification  of  (6)  will  not  be  completely  rigorous;  we  rely  on  the 
reader's  intuition:  Consider  n-space  as  partitioned  by  the  hyperplanes 
Hj,  . ..,  H^.  When  a  new  hyperplane  Hq  is  introduced,  how  many  regions  are 

affected?  Clearly, only  those  into  which  Hq  enters,  and  enters  to  the  extent 
that  it  has  an  (n-1 )-dimensional  intersection  with  them.  Each  such  region  is 
clearly  cut  into  two  pieces,  since  the  regions  are  simply  connected  (they  are 
bounded  by  hyperplanes)  and  the  separation  is  by  a  hyperplane.  Thus  the  new 
number  of  regions  is  just  the  old,  plus  the  nunber  of  (n-1 )-dimensional  regions 
into  which  Hq  is  cut  by  intersections  with  the  original  hyperplanes.  (To  say 
it  slightly  differently:  Each  piece  into  which  Hq  is  cut  acts  as  a  divider  in 
some  one  of  the  original  regions,  thus  adding  one  region  to  the  previous  total.) 
This  establishes  (6), 

Now  for  the  induction  step:  By  hypothesis,  the  middle  term  of  (6)  is 
equal  to  the  number  of  even-degenerate  subsets  of  T  H^ ,  ...,  H^  ^  minus  the 
number  of  odd-degenerate  subsets.  This  accounts  for  all  subsets  of  -|hq,  H,,...,H(nr 
that  don't  involve  Hq;  the  induction  step  will  be  complete  if  we  can  show  that 
the  number  of  even-degenerate  subsets  containing  Hq,  minus  the  number  of  odd- 
degenerate  subsets  containing  Hq,  equals  the  final  term  in  (6).  But  by  hypothesis, 
the  final  term  is  equal  to  the  number  of  even-degenerate  subsets  of 
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(w  — »  H0Hm}  ,  in  Hq,  minus  che  number  of  odd-degenerate  subsets. 

Suppose  we  nunber  the  H.  so  that  H_H, ,  . ..,  HnH  are  distinct,  while 

1  U  I  U  4  Up 

each  of  HqH  . ^ .  . ..,  HqH^  is  a  duplicate  of  one  of  the  first  p.  Now  consider 


any  subset 


n  j  |  ,  •••y  “Q  jjj  «*  uupu^ate 

{  Vi,'  Vi2’  -■  Vtk} 


where  each  i.  <  p.  The  intersection 
J  ~ 


(Vi,)(Vi2)-(Vik)-  Vi,Hi2  •••  \ 


(7) 


in 


Note  that  the  degeneracy  (odd  or  even)  of  the  subset  j  Ho^i  ’  H0H'  I 

^  1  1k  J 

(n-l)-space  is  the  same  as  the  degeneracy  of  the  subset  ,H.  H  >- 

L  H  *2  *k  -> 

in  the  original  n-space  (because  in  the  latter  case  the  spatial  dimension  is 
one  greater,  and  the  number  of  planes  is  one  greater,  while  the  intersection  -- 
and  its  dimension  --  remains  unchanged,  by  (7)).  We  have  proved  this  equivalence 
for  all  subsets  of  {Vi-  VP}  since  here  there  are  no  duplicates.  Thus, 

the  right-hand  side  of  (6)  equals  the  number  of  even-degenerate  subsets  of 
"C  H0,Hr  ***'  Hm  }  except  those  containing  HQ  and  at  least  one  of  H^, 

minus  the  number  of  odd-degenerate  subsets  with  the  same  exception.  If  we  can 
8 how  that  the  sun  corresponding  to  this  exception  is  zero,  the  proof  will  be 
complete. 


So  consider  H^j .  By  assunption,  for  some  r  <  p 


H0Hp+1 


H0Hr  * 


(8) 


Consider  all  subsets  of  {  H0>  H|>  •••>  Hm  }  which  contain  HQ  and  but 

none  of  H_in,  ...,  H^.  They  come  in  pairs;  for  tach  such  subset  containing 


p+2’  * 


Hr  there  is  one  identical  except  lacking  Hr.  Consider  the  intersections 
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corresponding  to  such  a  pair  --  call  them  Hq  II  and  Hq  H  .  ("H" 

for  some  "product"  of  hyperplanes).  Multiplying  (i.e.,  intersecting)  each  side 

of  (8)  by  H  H ,  we  see  that  the  intersections  are  equal: 
p+l 

"o'1  Vi  *  ".'W  <9) 

Thus  the  dimensions  are  equal.  And  since  the  manber  of  hyperplanes  in  the 
corresponding  subsets  differ  by  one  (Hf),  one  of  the  subsets  must  be  even- 
degenerate,  and  the  other  odd-degenerate.  Because  of  this  cancellation,  we  can 
strengthen  the  next-to-last  sentence  of  the  previous  paragraph:  The  right  side 
of  (6)  equals  the  number  of  even-degenerate  subsets  of  Hq,  Hj,  ..., 
except  those  containing  Hq  and  at  least  one  of  H^^*  H^,  minus  the  number 

of  odd-degenerate  subsets  with  the  same  exception. 

Now  we  repeat  the  argument  with  H^^*  then  again  with  H^  j,  and  so  on, 
until  the  exceptions  are  removed:  The  left  side  of  (6)  equals  the  right  side 
of  (6),  which  equals  the  number  of  even-degenerate  subsets  of 
minus  the  nunber  of  odd-degenerate  subsets,  QED. 
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IV.  BOUNDS  ON  THRESHOLD  GATE  REALIZABILITY 
by  R.  0.  Winder 

It  is  well  known*  [Winder-1],  [Cameron]  that  the  number  of  threshold 
functions  of  n  arguments  is  less  than 

B»  *  2Z  (2”;')  • 

i*o  ' 

Using  this  bound,  Cameron  [op  cit]  shows  that  the  number  of  n-argument  functions 
realizable  by  a  network  of  at  most  k  threshold  gates  is  (asymptotically)  less 
than  2^  .  As  Cameron  pointed  out,  this  implies  that  at  least  one  switching 
function  of  n  arguments  (and  probably  most  of  them)  requires  more  than  2n/ 
threshold  gates  for  realization.  The  purpose  of  this  note  is  to  generalize 
these  results  to  functions  incompletely  specified;  an  improvement  in  the 
asymptotic  bound  2n^  will  also  be  obtained.  Applications  in  character  recog¬ 
nition  and  self-organizing  systems  are  discussed. 

The  basic  bound  Bfl  is  derived  from  the  following  basic  lemma  (see 
[Cameron]  for  a  good  discussion  of  its  proof  —  the  proof  in  [Winder-4]  is 
virtually  identical,  but  less  well  explained): 

Lemma;  If  m  hyperplanes  are  passed  through  the  origin  of  an  (n+1)- 
dimensional  Euclidean  space,  the  space  is  divided  into  a  number  of 
regions  at  most  n 


The  bound  Bn  is  then  obtained  by  considering  an  (n+1 )-dimensional 

"realization  space"  —  the  space  consisting  of  points  "t  »  (a  ,  a .  a  ). 

o7  l7  7  n  7 

each  of  which  represents  the  realization  of  some  threshold  function  (a  bias 
and  n  weights).  (We  assume  a  ±  1  logic.)  By  taking  all  possible  choices  of 
sign,  we  consider  2  hyperplanes 


ao  i  a,  ±  a2  1  ...  ±  an  -  0. 


Two  points  in  the  realization  space  represent  the  same  function  if  and  only 


*D.  T.  Perkins,  D.  G.  Willis,  and  E.  A.  Whitmore,  of  Lockheed  (Missiles  and  Space 
Division),  have  not  published  their  early  results  on  this  problem. 
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if  they  ere  not  separated  by  any  of  these  hyperplanes.  Thus  the  regions, 
with  boundaries  defined  by  these  hyperplanes,  correspond  one-to-one  with 
threshold  functions.  Thus  setting  m  ■  2n  in  the  lesina  gives  Bfl. 

Now:  Suppose  we  select  exactly  m  out  of  the  2n  possible  input  combi¬ 
nations.  How  many  switching  functions,  no  two  of  which  agree  in  value  on 
all  m  of  these  points,  can  be  realized  by  a  single  threshold  gate?  Clearly, 
by  the  same  argument,  there  are  at  most  B®.  (Because  the  m  points  correspond 
to  m  of  the  hyperplanes,  and  again,  we  are  asking  how  many  regions  the  reali¬ 
zation  space  is  divided  into,  here  by  m  hyperplanes.) 


Bounds  on  B'  are  easily  obtained; 


n  ^ 

2  \  /m*1  l  »  2  I®"1  +-S-  + _ afe-O _ +  + _ ^ _  1 

/j  y  i  J  \  n  /L  +  m‘n  (m-n)(m-n+1)  ***  (m-n) . . . (m-1 ) J 


2  Mi;  +-D_  +  7aL  + 

\  n  J  |_  m-n  (m-n)*  (m-n, 

9  7  /m-l\  m-n 

\  n  /,  n  \  n  /  m-2n 


3  + 


] 


m-n 


(providing  n  <  m-n,  i.e.,m  >  2n,  which  we  henceforth  assume).  So 


Bm  <  2  1 ) (®- 2) • . • (m- (n- 1 ) ) (m-n)  .  m-n 

n  n!  m- 2n 


Assuming  m  >  3n  +  2  and  n  >  2,  which  we  do  henceforth,  we  take  four  of  the 
right-most  factors  above; 


+  j|-) 


2  n2+n-2-n2  n2(n-1) 

m  .  - — -  rn  -  — — •- 

n+2  n+2 


2 

<  m  . 
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So 


2a 


<  m 


(we  uae  the  relatively  sloppy  bound  mn  because  of  its  convenience;  sharper 
results  below  can  be  obtained  using  2m  /n!).  Summarizing: 


Theorem:  When  exactly  m  points  only  are  considered,  at  most 


essentially  different  functions  can  be  realized  with  a  single  thresh¬ 
old  gate. 

Rule-of-thunb:  When  m  >  n  log^ni,  single-gate  realizability  is 
improbable. 

Justification:  The  total  number  of  switching  functions  specified  on 
exactly  m  points  is  2m.  When  2m  >  mn,  we  have  that  all  such  functions  cannot 
be  single-gate  realized.  Because  of  the  rather  generous  approximations  made 
in  deriving  the  bounds,  it  seems  reasonable  to  say  that  by  this  point,  most 
functions  cannot,  in  fact,  be  single-gate  realized;  hence, the  rule-of-thumb. 
(Strictly  speaking,  we  can  only  say  that  when  2m/mn  >  a  ,  i.e.,m  >  n  log^m 
+  logj  a,  we  have  at  best  a  1/a  chance  of  realizing  a  random  function, 
specified  on  m  points.  Taking  a  -  1000,  for  instance,  however,  affects  the 
application  very  little.) 

An  example:  Suppose  we  have  a  Perceptron-Pappa- Pandemonium  type 
network  of  threshold  gates,  i.e.,a  large  number  of  randomly  chosen  threshold 
gates  (A  units)  accepting  inputs  from  some  "retinal"  field,  and  supplying, 
say,  n  ■  100  inputs  to  an  output  threshold  gate  (the  R-unit).  Although  the 
first  level  of  gates  is  fixed,  we  are  free  to  choose  the  weights  and  thres¬ 
hold  of  the  final  output  gate  as  we  please.  What  chance,  then,  have  we  of 
distinguishing  between  two  given  different  types  of  pattern  on  the  retinal 

field?  Applying  the  rule  of  thianb,  we  see  the  following:  If  there  are 

100  30 

m  ■  1000  or  more  (out  of  the  total  of  2  ~  10  )  input  configurations  in 

the  two  patterns,  upon  which  we  want  the  network  to  discriminate  reliably, 
we  probably  will  not  be  able  to  find  any  choice  of  weights  and  threshold 
that  will  work. 
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A  valid  objection  to  this  type  of  argument  is  the  following:  The 
actual  patterns  that  we  want,  in  fact,  to  discriminate  between,  are  not 
random  —  they  are  likely  to  have  natural  regularities  and  redundancies, 
which  may  make  them  more  easily  realizable  than  these  enumerational  sta¬ 
tistics  would  indicate.  However,  in  the  case  of  Perceptrons,  etc.,  note 
that  the  first  level  of  randomly  connected  gates  has  served  to  eliminate 
such  regularities!  The  Perceptrons,  etc ,f  therefore  are  at  the  mercy  of 
these  statistics;  tne  possibility  of  true  generalization,  where  surely 
m  »  1000  is  required,  is  remote  indeed. 

Up  to  this  point  we  have  been  considering  single-gate  synthesis. 

Suppose  we  now  investigate  the  capabilities  of  networks  of  threshold  gates. 
Let  k  be  the  ntanber  of  gates  in  a  given  network,  n  and  m  defined  as  before, 
relative  to  the  overall  function  produced  by  the  network.  Generalizing 
and  strengthening  Cameron's  [op  cit]  arguments,  we  can  prove 

Theorem:  When  exactly  m  points  only  are  considered,  a  network 
receiving  n  external  inputs,  and  consisting  of  k  threshold  gates, 
can  realize  at  most 

[nk  +  k(k-1  )/2 ] 
m 

essentially  different  functions. 

Proof:  We  assume  that  there  are  no  feedbacks  in  the  network,  so  that 
there  is  some  order  of  gates:  1st,  2nd,  ...,  kth.  The  1st  receives 
just  the  externally  provided  n  inputs,  the  2nd  receives  these  plus  the  out¬ 
put  of  the  1st,  etc.,  and  the  kc^  receives  the  basic  n  plus  the  output  of 
the  k-1  earlier  gates,  and  supplies  the  network  output.  On  the  m  specified 
input  configurations,  the  1st  gate  realizes  at  most  mn  threshold  functions. 

For  a  given  choice  of  function,  the  2nd  gate  is  also  presented  with  at  most 
m  specified  input  configurations,  and  so  can  realize  at  most  functions. 

And  so  on:  The  kc^  gate  can  realize  at  most  functions.  The  total 

number  of  functions  calculable  by  the  network  (with  regard  to  the  originally 
chosen  m  points)  is,  then,  at  most 

n  n+1  n+k-1  [nk  +  k(k-1 )/2 ] 

m  •  m  •  ...  .  m  ■  m  . 
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END  OF  PROOF. 


Rule-of -Thumb:  When 


m  <  [nk  +  k(k-1)/2]  log2m, 

a  network  of  k  gates  will  not  suffice  for  most  functions. 

Proof  is  as  for  the  previous  rule-of-thunb. 

An  example:  Consider  a  character-recognizing  system  with  n  -  100,  and 
suppose  m  -  8000,  i.e.,  8000  variations  on  a  given  character  are  to  be  recog¬ 
nized.  By  the  above,  then,  one  would  expect  to  require  k  ■  6  threshold  gates 
for  realization,  at  least.  Of  course,  here  the  well-behavedness  of  naturally 
arising  functions  can  be  expected  to  help;  the  bound  is  only  a  rough  approxi¬ 
mation  of  what  might  be  needed. 

Corollary:  For  completely  specified  functions  (m  «  2n),  if 


then  realizability  in  a  network  of  k  gates  is  improbable. 

Proof;  Substitute  m  ■  2n  in  the  above  rule-of-thunb  and  solve  for  k. 
The  bounds  hold  from  n  ■  4: 


and  above  n 


n 

k 

n 

k 

n 

k 

4 

1 

10 

8 

16 

77 

5 

2 

11 

12 

17 

109 

6 

2 

12 

18 

18 

154 

7 

3 

13 

26 

19 

218 

8 

4 

14 

37 

20 

305 

9 

5 

15 

54 

20,  the  bound  is  very  close  to  the  dominant  term; 


Trt-1 


"T~ 


(This  is  a  substantial  improvement  over  Cameron's  asymptotic  bound  of  2 


n/3 


) 
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V.  NETWORKS  OF  3- INPUT  MAJORITY  GATES 


We  report  here  on  an  introductory  exploration  of  synthesis 
methods  for  networks  of  3-input  majority  gates.  The  results  obtained 
were  largely  negative,  so  that  the  discussion  will  be  brief. 

A  3-input  majority  gate  (called  henceforth  a  3-gate)  realizes 
the  switching  function 

maj  (x,  y,  z)  ■  xy  +  xz  +  yz  ■  (xyz) 

denoted,  in  this  report,  by  the  parenthetical  closure  of  three  arguments 
(as  on  the  right).  The  synthesis  problem  is:  Given  an  arbitrary  switching 
function,  find  a  network  of  3-gates  which  realizes  it.  Minimality  criteria 
may  include  the  number  of  gates  and  the  depth  of  the  network  (i.e.,  the 
maximal  stage  delay). 

It  was  decided  to  study  first  functions  of  only  four  arguments. 
Computer  programs  (for  the  RCA  SOI )  were  written, debugged,  and  run  to 
experisient  with  various  trial  synthesis  procedures.  The  first  of  four 
programs  written  converted  the  Harvard  Computation  Laboratories  list  of 
238  four-argusent  PN  symmetry  types  into  a  list  of  easily  manipulatable 
truth  table  representations  in  the  RCA  501 .  Subsequent  programs  used  this 
data,  so  that  a  trial  procedure  could  be  tried  out  (in  effect)  on  every 
one  of  the  65,536  four-argument  switching  functions. 

Program  niaaber  two  was  written  to  try  out  a  simple  "synthesis- 
by-expansion"  idea,  just  to  see  how  it  works.  The  expansion  theorem: 

Px  “  (yx  Py>  (y*  Py>  P 

was  taken  from  [Cohn-Lindaman] .  Its  effect  is  to  replace  the  problem  of 

synthesizing  P  by  the  problems  of  synthesizing  P-  and  P  separately,  where 

x  y  y 

y  can  be  chosen  so  as  to  make  these  problems  simpler,  —  all  at  the  expense 
of  three  3-gates.  Program  nunber  two  made  the  following  ten  substitutions 
for  x  and  y: 

(x,y)  -  (w, 0),  (w,x),  (w,y),  (w,z), 

(x, 0),  (x,y),  (x,z),  (y,0),  (y,z),  (z,0). 
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Thus  in  each  case  and  P-  had  only  three  arguments,  with  known  optimal 
realizations.  The  number  of  gates  required  for  each  substitution  for  each 
function  type  was  printed  out.  Choosing  the  best  of  the  ten  for  each  type, 
an  average  of  6.2  3-gates  was  required;  this  is  surprisingly  good,  although 
far  from  the  optimal  networks  obtainable.  At  most  four  stages  are  required 
by  this  expansion  —  all  four  were  usually  needed.  The  experiment  showed 
that  all  ten  of  the  basic  expansions  should  be  considered  —  the  range  from 
best  to  worse  over  this  choice  was  large  (e.g.,  from  6  to  11  gates  were 
required,  depending  on  the  choice  of  expansion  variable) . 

Program  number  three  was  written  and  run  to  find  all  two-stage 
3-gate  realizations  that  exist  for  4-argument  symmetry  types.  A  modified 
form  of  the  method  of  [Miiller-Winder ]  was  used.  Of  the  238  symmetry  types 
52  were  found  so  realizable.  These  results  gave  known  optimal  networks, 
then,  for  a  large  ntmber  of  the  functions  we  were  studying. 

Program  number  four  was  our  first  serious  try  at  a  general 
synthesis  method;  it  was  intended  to  apply  to  functions  of  any  number  of 
arguments.  The  algorithm  is  as  follows: 

(i)  Initialize  by  listing  all  functions  of  the  given  input 
variables  that  can  be  realized  by  a  single  3-gate.  Then  go  through  the 
following  steps  repetitively  until  a  realization  is  found,  or  patience  is 
exhausted. 

(ii)  Choose  the  k  functions  in  the  list  which  are  most  similar 
to  the  given  function  to  be  realized  (i.e., which  agree  for  the  most  input 
specifications),  [if  any  agree  exactly,  exit.] 

(iii)  Take  all  possible  triples  from  this  set  of  k,  forming  the 
majority  of  each  triple,  and  listing  the  resulting  realized  functions  to 
replace  the  previous  list  of  candidates.  Return  to  step  (ii). 

The  nvanber  of  gates  in  the  final  network  (when  one  is  obtained) 

m  j  |  a 

is  (3  -1)/2,  where  m  recursions  were  required.  The  results  of  the  appli¬ 

cation  of  this  algorithm  to  the  238  4-argisnent  symmetry  types  were 
disappointing.  Unfortunately,  the  experimentation  ended  before  the  reason 
for  this  was  satisfactorily  established  (bugs  in  the  program?  faulty  intuition? 
too  small  a  k  used?  —  we  used  k  ■  10.  etc.). 
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To  complement  the  positive  results  of  program  number  three  (the 

2- stage  realizations)  and  to  round  out  the  study  of  four-argument  functions, 
a  long  and  tedious  hand  application  of  modifications  of  the  [Miiller-Winder ] 
method  was  made.  Instead  of  treating  each  of  the  238  Harvard  FN  symmetry 
types,  we  used  a  new  and  better  classification  method  of  [Goto-Takahasi]. 

By  this  method,  only  83  types,  called  SD  (for  self  dual),  are  needed.  The 
hand  calculations  involved  first  a  derivation  of  a  set  of  83  SD  repre¬ 
sentatives  (which  are  listed  in  the  appendix)  and  then  a  realization  of  each, 
in  an  attempt  to  obtain  3-stage  realizations  for  all  4-argunent  functions. 
(The  average  number  of  gates  needed  was  6.9,  2.8  stages  average.)  All  but 
two  SD  types  were  3-stage  realizable  (the  realizations  are  listed  in  the 
appendix.)  The  two  exceptions  were  the  parity  function  and  the  "almost- 
parity"  function  (differing  from  the  parity  function  at  exactly  one  point). 

It  was  found  possible  to  prove  that  neither  of  these  functions 
are  3-stage  realizable,  by  the  following  argument:  If  a  function  f  is 

3- stage  realizable,  then  f  -  maj(g,h, i),  where  each  of  g,  h,  and  i  are 

2- stage  realizable.  The  results  of  program  nianber  three  identified  the  19 
SD  types  which  are  2-stage  realizable;  these,  then  were  the  only  candidates 
for  g,  h,  and  i.  The  next  step  in  the  argument  is  to  note  that  since  f 
agrees  with  the  majority  of  g,  h,  and  i  at  each  of  the  2n  input  points  (there 
are  four  arguments,  and  the  representative  SD  types  have  five  arguments,  so 

n  ■  5),  the  sun  of  the  number  of  agreeing  points  between  f  and  g,  f  and  h, 
and  f  and  i,  respectively,  must  be  at  least  2  x  2n.  By  considering  the  17 
cases,  however,  it  was  found  that  none  agreed  with  the  parity  function  in 
more  than  20  positions,  so  the  sum  could  not  exceed  3  x  20  <  2  x  2*’.  Thus 
the  parity  function  is  not  3-stage  realizable.  Similarly,  only  two  2-stage 
realizable  types  were  found  which  agreed  with  the  almost-parity  function  in 
more  than  20  places;  and  they  agreed  in  only  22  positions,  so  both  would  be 
needed  to  realize  it.  But  by  [Miiller-Winder ]  methods  they  did  not  suffice. 
Thus  neither  the  parity  function  nor  the  almost-parity  function  can  be 

3- stage  realized  by  3-gates.  Since  we  know  all  functions  2-stage  realizable 
(by  program  number  three),  we  have  proved  that  the  realizations  listed  in 
the  appendix  are  minimal  in  number  of  stages  (if  not  number  of  gates). 
Stmmarizing  (these  data  were  reported  in  [Winder-5]): 
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Minimal  Nunber 
of  Stages 

Nunber  of 

SD  types 

Number  of 
Functions 

0 

1 

10 

1 

1 

80 

2 

17 

1 0, 260 

3 

62 

55,152 

4 

2 

34 

83 

65,536 

General  conclusions:  The  study  has  only  just  begun.  Expansion 
methods  deserve  more  attention.  Work  on  more  ambitious  general  programs 
should  commence  with  an  analysis  of  program  number  four,  and  further  experi¬ 
mentation  with  it.  The  realizations  obtained  by  hand  should  be  analyzed. 
Comparison  with  the  methods  of  [Akers-3]  should  be  made.  (Akers  has  run 
ten  of  the  83  types  with  his  program,  obtaining  two  realizations  definitely 
better  than  our  hand  calculations,  two  realizations  definitely  worse,  two 
identical,  and  four  that  use  fewer  gates  but  more  stages.  Over  these  ten, 
he  averages  4.7  gates,  3.4  stages,  as  compared  with  5.9  gates  by  the  simple 
expansion  program  (mmtber  two)  and  somewhat  less  than  four  stages,  and 
compared  with  5.5  gates  by  hand,  and  2.7  stages.) 
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VI.  TWO  METHODS  OF  THRESHOLD  GATE  SYNTHESIS 
by  R.  0.  Winder 


The  optical  recognition  of  characters  is  a  commercially  important  problem;  much  of 
what  is  said  below  is  significant  in  this  area.  Pattern  recognition,  a  generalization  of 
character  recognition,  plays  many  roles  in  artificial  intelligence.  As  Minsky*  sees  it, 
for  instance,  one  wants  a  computer  to  categorize  the  various  problems  presented  to  it 
into  patterns,  in  order  to  choose  suitable  methods  of  solution.  To  do  this,  the  computer 
determines,  for  each  incoming  problem,  whether  or  not  it  satisfies  each  of  a  list  of 
properties .  (The  analogous  procedure  is  usually  followed  in  optical  character  recognition. ) 
if  we  call  this  list  of  properties  an  input  vector,  then  the  final  step  in  the  process  is ; 

Given  an  input  vector,  to  determine  into  which  category  the  given  problem  is  to  be  placed; 
each  category  has  a  set  of  associated  methods  to  be  used  in  solving  the  problem.  Without 
loss  of  generality,  we  will  restrict  ourselves  to  the  question  of  determining  whether  or 
not  the  given  input  vector  belongs  to  a  single  given  pattern. 

This  reduces  the  problem  to  switching  theory:  To  accomplish  the  last  step  in  pattern 
recognition,  a  certain  switching  function  of  the  input  vector  must  be  computed.  (If  the 
input  vector  belongs  to  the  pattern,  the  value  of  the  function  for  that  vector  is  1,  other¬ 
wise  0.)  Theoretically,  switching  theory  could  be  used  to  realize  such  a  function.  How¬ 
ever,  because  the  number  of  inputs  is  often  far  too  large  to  be  handled  by  conventional 
switching  theory,  and  because  the  function  cannot  be  completely  specified  in  practice,  the 
following  procedure  is  often  followed  to  design  a  decision  network.  (We  follow  Minsky’s 
excellent  discussion  of  the  matter.  *) 


A.  BAYES  NETS 

We  make  several  assumptions.  First,  we  "assume  that  the  situation  is  basically 
probabilistic"  (Minsky).  Next,  we  assume  that  we  know,  or  can  estimate,  certain 
conditional  probabilities:  the  probability  of  the  i^1  property  holding,  given  that  the  input 
vector  belongs  to  the  pattern.  Finally,  we  assume  that  these  probabilities  are,  in  a 
sense  described  below,  independent.  Now  we  can  design  a  system  which  is  able  to 
compute  the  probability  that  a  given  input  vector  should  be  assigned  functional  value  1  or 
0  (in  or  out  of  the  pattern).  When  several  possible  patterns  are  being  considered,  the 
system  might  classify  by  choosing  the  pattern  with  the  highest  probability.  More 
generally,  we  specify  levels  of  confidence,  so  that  the  machine  may  fail  to  classify 
certain  unusual  problems;  in  our  case  of  a  single  pattern,  a  confidence  level  A  >  1/2 
would  be  used. 


*  M.  Minsky,  "Steps  Towards  Artificial  Intelligence",  Proc.  Inst.  Radio  Engrs. 
Vol.  49,  pp.  8-30,  January  1961. 
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More  explicitly,  suppose  the  inputs  are  associated  with  variables 


x  =  <V  x2 .  xn> 

and  a  particular  state  of  the  inputs  is  represented  by 

4  =  l »  ^  2* .  ^  n^  * 


and 


We  are  assuming  that  we  know  the  probabilities 

Pj  =  Pr(xt  =  1  I  f(x) 

rj  =  Pr(xt  =  i  I  f(x) 

pQ  =  Pr(f  (x)  =  l). 


1) 

0) 


From  these  we  obtain  the  related  probabilities 

1  -  Pj  =  Pr(Xj  =0  I  f  (x)  =  1) 

1  -  r.  =  PriXj  =  0  I  f(x)  =  0). 

The  independence  assumption  is  that 

_  _  _  n 

Pr(x  =  4  I  f(x)  =1)  =  n  Pr(x  =  4  I  f(x)  =  1) 

i  =  l 


and  similarly  with  f  (x)  =  0. 

Now  we  use  the  Bayes  law,  writing 

Pr(f (x)  =  1  and  x  =  4)  =  Pr(f(x)  =1  I  x  =  7)  •  Pr(x  =  "4) 

=  Pr(x  =7  I  f(x)  =  1)  •  Pr(f (x)  =  l). 


Thus 


Pr(f ( 4)  =  1)  =  Pr(f (x)  =  1  I  x  =  4) 
=  Pr(x  =  4  I  f(x)  -  1 ) 


Pr(f(x)  =  1) 

pr(x  =  T) 
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Three  alternatives  are  available:  1)  We  can  write  Pr(x  *  4 )  in  terms  of  the  probabilities 
given  US’11,  yielding  somewhat  more  complicated  rules  than  we  obtain  below,  2)  We  can 
follow  Minsky's  discussion,  taking  at  this  point  a  maximum  over  the  quantities 

Pr(x  =  7  I  f(x)  =  1)  =  n  PriXj  =  4.  I  f(x)  =  1) 


for  the  various  patterns  (f's),  or  3)  We  can  assume  at  this  point  that  the  |  are  equiprobable, 
so  that  Pr(x  =  4  )  is  independent  of  4  .  To  simplify  the  illustration  and  future 
computations  we  chose  the  last  alternative. 

If  Pr(f(4)  =  1)  is  calculated  tojie  greater  than  some  confidence  level,  then,  we 
will  ask  the  machine  to  guess  that  f  ( 4 )  =  lji.e. ,  to  classify  an  incoming  problem  into 
the  corresponding  category).  Otherwise,  f  ( 4 )  =  0.  We  convert  this  calculation  into 
the  characteristic  threshold  function  form  as  follows : 


where 


Pr(f(£)  =  1)  2  A 

Pr(x  -  4  I  f(x)  =  1)  2  A 


Pr(x  =  4) 
Pr(f(x)  =  1) 


if  and  only  if 
if  and  only  if 


II  Pr(x.  =  4  I  f(x)  =  1)  2  A 
i  1  1 


Pr(x  -  4) 
Pr(f(x)  =  1) 


if  and  only  if 


n  Pr(xi  *  1  W  =  »  >  , 

i  Pr(x  »  0  I  f(x)  -  1) 


(for  appropriate  0) 
if  and  only  if 


l 


log 


Pr(xj 

Pr(Xi 


li  I  f  (x)  =  1) 

"0  I  f  (x)  =  1)  “  log  0 


1  *.  iT- 


if  and  only  if 


a 


i 


log 


Pr(Xj  =  1  |  f(x)  =  l) 

Pr(x  j  =  0  I  f(  x)  =  1) 


and  T  =  log  6 .  T  can  be  calculated  from  the  various  probabilities  that  we  know,  as 
are  the  aj ,  but  the  general  philosophy  here  is  to  generate  the  as  specified,  and  to 
pick  the  T  so  as  to  optimize  performance. 

*  M.  E.  Maron,  'Design  Principles  for  an  Intelligent  Machine",  IKE  Trans.  Inf.  Theory, 
Vol.  IT-8,  pp.  179-185,  September  1962.  ~  “  '  “ 
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We  will  now  discuss  *!v-  allowing  question:  If  we  intend  to  use  a  threshold  func¬ 
tion  for  the  recognition  procedure,  does  the  above-stated  point  of  view  provide  the  best, 
or  even  good,  sets  of  weights  a  ? 

Saul  Amarel  (RCA  Laboratories)  has  suggested  that  the  independence  assumption  is  a 
very  severe  one,  and  that  only  a  small  proportion  of  threshold  functions  might  actually  be 
realized  by  the  above  procedure.  This  observation  has  motivated  the  investigations  re¬ 
ported  below  —  indeed,  using  the  same  data,  a  simpler  rule  will  be  shown  to  work  better, 
and  allow  a  startling  improvement  ;  a  nonprobabilistic  switching  theory  viewpoint  is  applied 
to  obtain  these  results. 


We  facilitate  a  comparison  by  considering  the  following  situation:  All  2°  of  the  £ 
are  given  equal  probability  of  occurring,  and  each  f  (£)  is  specified.  We  assume  that  f 
is  a  threshold  function.  Probabilities  are  obtained  simply  by  counting  frequencies: 

Let  f  x  j  be  the  restriction  of  f  obtained  by  setting  x  j  =  1 ,  f  .  obtained  by  setting  x  \ 

-  0,  and  let  m(f)  be  the  number  of  different  £  such  that  f(£)  =  1. 

suppose  the  2n  possible  £  are  mapped  as  in  the  diagram  (f  Ci)  =  l 
area,  0  otherwise): 


Schematically, 
in  the  shaded 


=  1 


=  0 


If  A,  B,  C,  and  D  are  the  number  of  input  vectors  £  in  the  four  regions,  respectively, 
then  A  =  m(fx),  B  =  m(fx),  C  =  m(fx),  and  D  =  m(fx).  Note  that  m(f)  =  B  + 
D  =  m(fx)  +  m<fjj),  m(fx)  +  m(fx)  =  2n”1,  etc.  Since  the  £  are 
equiprobable,  we  have 


m(f) 

ixj) 
m(T) 


m(fXi) 


m(f) 


.n 
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Theorem:  The  Independence  assumptions  hold  only  for  constant  functions  and  func¬ 
tions  which  are  the  identity  (or  complement)  in  some  argument. 


Proof:  If  f  is  constant  (0  or  1),  the  independence  is  trivial;  so  suppose  f  is  not  a 
constant  function.  By  simple  properties  of  threshold  functions,  we  now  know  that  we 
can,  without  loss  of  generality,  assume  that  f  (0)  =0  and  f  (1)  =1 ,  where 

0.  -  (0,  0,  •  •  • ,  0) 

1  =  (1,  1 . 1)  . 

Thus 


Pr  (x  =  1  |  f  (x)  =  0)  =  0, 


so  that  if  the  independence  assumptions  hold. 


0  =  n  r 

i  * 


n 

i 


m(f  ) 
_  i 
m(f) 


But  one  of  the  factors  must  be  zero,  so  for  that  i, 

m(f)  =  2n  -  m  (f)  =  2n  -  [  m  (f  )  +  m  (f-  )  ] 

xi  xi 

=  2n-m(f-  )  a2n  -  2n_1  =  2D_1  . 
i 


On  the  other  hand 

Pr  (x  =  0  I  f(x)  =  1)  =  0. 

So,  by  the  independence  assumption 

m(f*  > 

0 

Again  some  factor,  say  the  j  ***, 

m 

"j 


=  n  (i  -  p  >  *  n<i  - 

i  i 


m(f) 


must  be  zero: 


(f)  =  m(f  ) 


(1) 


(2) 
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But  now,  from  Eqs.  (1)  and  (2), 


2n_1  ==m(f)  =  m(f  )  £  2n-1 , 

Xj 

so  that  all  the  terms  are  equal.  Thus  for  some  j,  fxj  must  be  the  constant  1.  Also, 
since  m  (f  -  )  =  m  (f)  -  m  (f  )  =  0,  f  -  must  be  the  constant  0.  Thus  f  (x)  =  x.  . 

xj  xj  xj  1 

(END  OF  PROOF.) 

Thus  we  have  shown  that  for  airy  interesting  threshold  function,  with  equiprobable 
input  combinations,  the  independence  assumptions  in  fact  do  not  hold.  The  theorem  can 
probably  be  extended  to  more  general  input  distributions,  -  possibly  to  arbitrary  distri¬ 
butions.  This  substantiates  S.  Amarel'  8  conjecture,  and  suggests  that  the  Bayesian 
approach  can  be  improved.  In  the  next  section  we  consider  a  different  viewpoint. 


B.  GEOMETRIC  SYNTHESIS 

A  standard  switching-theoretic  geometric  interpretation  of  threshold  functions  is 
as  follows:  The  inputs  take  on  values  ±1,  the  input  n-tuples  |  are  mapped  correspond¬ 
ingly  onto  the  2n  corners  of  a  2  x  2  x  ...  x  2  cube  centered  on  the  origin  of  a  Euclidean 
n-space.  For  a  given  f,  those  corners  (or  vertices)  for  which  f  ( £ )  =  1,  sure  called  1- 
vertices;  if  f  ( |)  =0,  the  vertex  is  a  0-vertex.  An  (n  +  l)-tuple  aQ,  a^,  ...,  of  real 
numbers  is  a  realization  of  f  when  the  hyperplane  u 


a 

o 


+  a. 


V 


•  •• 


+  a  x  =  0 
n  n 


separates  the  cube  so  that  O-vertices  lie  on  one  side,  1-vertices  on  the  other;  when  such 
a  hyperplane  exists,  f  is  a  threshold  function.  (Note:  We  still  use  0  and  1  for  outputs.) 


In  order  that  we  can  treat  the  threshold  in  the  same  manner  as  other  weights,  and 
so  that  the  hyperplane  will  pass  symmetrically  through  the  origin,  we  add  a  new  dimen¬ 
sion  and  variable  Xq;  when  x*,  =  +  1  we  define  the  new  function  to  agree  with  our  original 
function;  when  Xq  =  -1,  we  define  the  new  function  to  agree  with  the  dual  of  the  original. 
(This  is  the  self-dualization  idea  of  [ Goto-Takahasi] . ) 

Now:  An  appropriate  normal  form  for  the  equation  of  such  a  hyperplane  is 


x  cos  a  +  x.  cos  a.  +  x„  cos  +  ...  +  x  cos  a  =  0, 
ooll2  2  n  n  ' 


where  the  ai  are  direction  angles  between  a  normal  to  the  plane  and  the  corresponding 
coordinate  axes.  We  will  relate  direction  angles  to  area  on  an  n-sphere,  and  then  area 
on  the  n-sphere  to  the  m  (fxj ).  by  straightforward  geometric  arguments,  described  and 
sketched  in  Fig.  1. 
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1 

I 


f  1 


Fig.  1  Sketch  of  n-sphere 


Suppose  we  pasB  an  origin-centered  (n+  l)-sphere  of  radius  V(n+1)  through  the 
2n  +  1  vertices  of  the  "cube"  which  we  have  defined  in  E  n  +  *  .  For  concreteness,  let 
us  consider  now  a  particular  coordinate  (say  x)  which  we  consider  "up".  (We  should 
speak  of  an  x  i  ;  "i"  is  dropped  for  convenience.  We  want  to  find  a  =  cos  a  for  this 
x. )  All  the  other  axes  define  a  hyperplane,  which  we  call  "horizontal".  If  we  pass 
hyperplanes  parallel  to  this  one,  at  a  distance  1  "above"  and  1  "below",  we  have  isolated 
the  regions  where  x  =  +1  and  x  =  -1,  respectively;  this  is  where  f  x  and  f  x  are 
represented.  We  will  consider  only  f  x  below;  by  our  construction,  fx  is  just  its  dual. 
Within  the  n-dimenslonal  region  containing  fx  ,  the  original  (n+l)-sphere  defines  an 
n-sphere  by  intersection.  Furthermore,  let  us  imagine  our  separating  hyperplane  passed 
through  the  figure;  it  will  divide  the  n-sphere  into  two  parts  (corresponding  to  O-vertices 
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and  1-vertices).  Now.  since  the  2n  vertices  are  spread  evenly  over  the  surface  of  the 
n-sphere,  the  ratio  m(fx)/m(fx)  is  an  estimate  of  the  ratio  of  areas  cutoff  by  the  sepa¬ 
rating  hyperplane.  For  greater  convenience,  we  define  the  ratio 

(  Area  on  1 -vertex  side  in  upper  n-sphere)  -1/2  (Area  of  n-sphere) 

1/2  (Area  of  n-sphere) 


m(f  )  -2  n  ” 1 
x 

^  —  —  •  -•  — 

2"-1 

(r  will  have  the  same  sign  as  the  final  weight;  r  =  0  when  f  is  insensitive  to  x. ) 


We  assume  that  r  is  known  by  counting  (or  sampling)  vertices;  we  see  next  that  this 
ratio  determines  the  distance,  D,  which  separates  the  separating  byperplane  (as  induced 
in  the  upper  n-sphere)  from  the  origin  (of  the  upper  n-sphere).  With  a  bit  of  trigonometry, 
it  can  be  seen  that 


dr  = 


J  2  n-2 

2Wn-l  (Vn*D  > 


dD 


wn 


n-1 


Vn-D2 


(where  is  the  area  of  the  unit  n-sphere),  so  that 


dD 

dr 


*  r 


2  u 


n-1 


3-n 

2 


Set 


k  = 


Vn  w 

n 


2  u 


n-1 


Initial  conditions  are  that  r  =  0  when  D  =  0  (and  r  =  1  when  D  =  Vn)- 


But  if  we  know  D  (in  terms  of  r),  then  we  can  determine  the  angle  0  between  the 
separating  hyperplane  in  (n+  l)-space  and  the  x-axis  —  we  have  a  right  triangle  with 
adjacent  side  1,  opposite  side  D.  The  direction  angle  a  is  the  complement  of  0  , 
so  that 


a  =  cos  a 


sign  (D) 
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(We  drop  the  "sign  (D)".  but  in  practice  it  is  used,  of  course,  to  determine  the  sign.') 
Solving  for  D: 


D 


2 


Now  we  can  calculate 


da  da  d  D  2 

d?=  dDd?  ‘  <D+1>  •  k<A- 


3  -n 
2 


n/2 


=  Ml  -  a  ) 


„  n+1  2v 

(i-  —  a  , 


3  -n 
2 


(after  substituting  and  rearranging).  Noting  that  a=r=0atD=0,  we  calculate  an 
expansion  for  a  around  r  -  0: 

.5 


a  =  kr-k3(2+-)  4-+k5(22  +  —  +  ^~)  — 
'  n'  IT  x  n  n2  '  120 


The  constant 


k  = 


Vn" 


]fn  2it 


n/2 


F  <7> 


2  w 


n-1 


2ir 


n-l 

2 


V^rl5r> 

2r<f> 


Vz  a.  n- 


r<^> 


«  (Wallis'  Theorem). 


If  we  take  out  the  dummy  x  and  divide  all  weights  by  k  •  2  _n  and  set  k4  (2  +  3/n)/6  \ 


we  obtain 


ai  =  l  m  (f  )  -m  (f-  )  ]  -  V  [  m  (f  )  -  m  (f  -  )  ]  3  4  . . . 
i  Xi  xi  x  i 

and  threshold 

T’=[2n_1  -m(f)  )  -X  [  2n_1  -  m  (f )  ]  3  4-  . . . 

This  threshold  is  for  a  ±  1  system;  using  a  standard  rule,  the  threshold  for  a  (0,  l) 
system  (as  defined  in  the  introduction)  is  then 


T  -  1/2  (T '  4  a  +  a  +•  . . .  4a). 

i  i  n 
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Experience  in  the  application  of  these  rules  has  been  very  good:  Often  the  first  approxi¬ 
mation,  ai  =  m  (f  x  j[  )  -  m  (f  x  i  )  and  T'  =  2  n  " 1  -  m  (f).  realizes  given  threshold  func¬ 
tions.  For  all  functions  tried  (including  a  complicated  ten -argument  example),  it  was 
easy  to  find  a  V  so  that  the  second  approximations  worked;  the  values  were  very  close  to 
the  predicted  \.'s.  Randomized  experiments  on  a  computer  are  planned  to  check  the 
general  worth  of  the  second  approximation,  to  determine  the  variation  of  X.  from  function 
to  function  for  a  given  n,  and  to  determine  a  good  value  of  X  as  a  function  of  n.  (X.=  0.  0008 
was  a  good  value  for  n  =  7,  X.  =  0. 00001  for  n  =  10. ) 

An  important  note:  Chow  [  Chow-2  ]  has  shown,  along  with  other  theoretically  very 
interesting  results,  that  the  signs  and  relative  magnitudes  of  the  a^  assigned  by  the  first 
approximation  a^  =  m(f  X;)  -  m(fXj  )  are  always  correct. 


C.  COMPARISON  AND  COMMENTS 

Theorem:  The  Bayes  approach  produces  a  set  of  weights  worse  than  does  the  first 
geometric  approximation,  described  above. 

Proof:  Assume  that  the  weights  should  all  be  positive;  if  they  are  not,  suitable  com¬ 
plementations  make  them  so.  It  is  clear  from  experience  with  the  first  approximation, 
and  follows  from  the  fact  that  a  better  approximation  subtracts  a  cubic  term  from  the 
first,  that  the  first  approximation  gives  exaggeratedly  high  first  estimates  to  the  larger 
a  4  ;  in  other  words,  for  a .  >  a^ ,  the  ratio 

m(f  )  -  m(f-  ) 

i _ i_ 

m(f  )  -  m(f-  ) 

Xj  j 

is  too  high  (for  the  best  chance  of  realization).  Let  us  abbreviate,  putting  =  m  (f  ^  ), 
and  m  =  m(f).  Now  we  will  show  that  the  Bayesian  weights  x i 


at  =  log 


m.  ,  m 

i/m  ,  i 

=  log 


1  -  m 


i/m 


m  -  m . 


are  worse  yet,  because  for  mj  >  , 


m , 


log 


m  -m . 
i 


m , 


log 


m-m , 


mt  -  (m - m ^) 
m  j  -  (m  -m  ^ 


m . 


•m/2 


m  -  m/2 
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which  is  already  too  large.  We  know  that  0  <  mj  <  m  i  <  m,  and  we  drop  the  equality 
possibilities,  where  the  Bayes  weights  blow  up.  Furthermore,  because  of  the  positivity 
assumption  (and  [Chow-2]  ,  for  example).  m(fxj  )  >  i.e.  mj  >m-mj,so 

mi  >  m/2.  Putting  y  =  and  x  =  21,  we  want  then  to  prove:  For  1/2  <  x  <  y  <  1, 

m  m 


log 


JL_ 

IzJL 


>  JLZIH. 

x  -  1/2  ' 


Since  log 


_x _ 

1  -x 


>  0  and  x  -  1/2  >  0,  we  need  only  show  that  the  function 


g(x.y)  =  (x  -1/2)  log  -  (y  -1/2)  log-j^ 


is  positive  in  the  range  R:  1/2  <  x  <  y  <  1.  But 


and 


9g 

9y 


(x  -1/2)  • 


hi.  .  Ti_ 

y  L1-y 


+ 


_y _ * 

(i-y)2 


-  log 


X _ 

1  -x 


_  (x  -1/2) 

y  (i  -y) 


-  log 


_x _ 

1  — X 


_9_  _9g_  _1 _ 

9x  ay  y(l-y) 


i  _  (y-x)  (y+x-i) 
x(l-x)  y(l-y)x(l-x) 


which  is  positive  in  R.  Furthermore 

Ml 

x=  1/2 

(the  left  boundary  of  R),  so  that  is  positive  in  R.  (There  are  no  discontinuities. ) 
But  ^ 


g(x,x)  -  0 

(the  lower  boundary  of  R),  so  that  g  is  positive  in  R  (again  no  discontinuities),  as  we 
wanted  to  show.  Thus  the  ratio  of  larger  weights  to  smaller  weights  is  greater  in  the 
Bayesian  approach  than  in  the  first  geometric  approximation,  where  it  is  already  too 
high.  (END  OF  PROOF.) 
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We  have  shown  that  the  Bayesian  approach  is  poor  tor  the  synthesis  of  threshold 
functions,  assuming  equiprobable  inputs  and  complete  specification.  It  should  be  clear 
that  neither  of  these  are  important  in  geometric  synthesis,  however:  Probabilities  of 
inputs  play  no  role,  and  the  areas  can  be  estimated  from  samples,  when  the  function  is 
incompletely  specified.  (Use  a  weighting  density  function  so  that  closely  packed  sample 
points  have  their  proper  effect  relative  to  sparsely  packed  ones. )  Whether  the  Bayesian 
approach  is  inferior  to  the  geometric  in  the  case  of  nonthreshold  functions,  has  not 
been  investigated.  (Here  probabilities  mean  something  —  the  problem  is  to  minimize  the 
probability  of  error,  say,  when  a  threshold  function  is  used  as  an  approximation. ) 

As  an  example  of  these  ideas,  consider  the  function  expressed  by 

A  [  B  (C  +  D+  E)  +  CD (E  +  F)  +  (C  +  D)  EF  ]  +BCD(E  +  F), 
which  has  (minimum  integral)  realization 

7,  6,  4,  4,  3,  2,  threshold  16. 

The  weights  generated  by  the  Bayesian  approach  are 


i  17 

log  T 


i  15  .  15  i  I4 
log  — ,  log  —  ,  log  — 


log 


li 

9 


or  (dividing  through  by  .  08) 

10.04,  6.66,  4.15,  4.15,  3.04,  2.00. 

With  these  weights,  at  best  62  out  of  the  64  vertices  can  be  separated  properly  (thresh¬ 
old  19.2).  Note  the  characteristic  manner  in  which  higher  weights  are  exaggerated,  as 
compared  with  the  minimal  realization. 

The  first  geometric  approximation  is 

19-3,  17-5,  15-7,  15-7,  14-8,  13-9,  threshold  32 


or  (dividing  by  2) 

8,  6,  4,  4,  3,  2,  threshold  16. 

This  realization  also  fails  on  two  vertices;  if  optimization  by  varying  the  threshold  is 
allowed,  the  same  weights,  with  threshold  17,  fails  only  on  one  vertex.  Any  (positive) 
X.  up  to  0. 004  will,  in  the  second  geometric  approximation,  give  a  realization  of  the 
original  function;  the  theoretically  calculated  value  of  0. 0034  works  fine. 
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A  final  comment:  The  iterative  synthesis  of  Mattson*  apparently  begins  with  our 
first  geometric  approximation  as  initial  trial  (although  no  rigorous  basis  is  suggested). 
Mattson's  idea  is  to  optimize  on  one  weight  (or  the  threshold)  at  a  time.  As  with  the 
Bayesian  approach,  however,  threshold  functions  will  usually  not  be  realized.  J. 
Sklansky  of  RCA  Laboratories  raised  this  issue  by  constructing  a  "pathological"  plane 
which  almost  realizes  A(B  +  C),  but  which  cannot  be  transformed  to  realization  by 
changing  any  one  coefficient.  The  example  discussed  above  was  subsequently  chosen 
to  illustrate  that  even  with  a  very  good  initial  try  (in  fact,  the  try  which  I  believe  Matt¬ 
son  intended),  namely 


8,  6,  4,  4,  3,  2,  threshold  6 
(in  the  ±1  system),  which  misses  only  the  1-vertex 

ABCDEF:  -8  +  6 +4 +4-3  +  2  =  5  <  6, 

Mattson's  iteration  wont  converge:  If  a  change  in  some  weight  is  to  Improve  the  reali¬ 
zation,  it  will  clearly  have  to  be  a  decrease  in  the  weight  a  of  A,  an  increase  in  b  or  c 
or  d,  a  decrease  in  e,  an  increase  in  f,  or  a  decrease  in  T.  But  a  decrease  in  a  loses 
the  1 -vertices  ABCDEF,  ABCDEF,  and  ABCDEF  at  the  same  time  it  gains  ABCDEF  — 
this  is  no  progress.  Similarly  increasing  b  loses  J. -vertex  ABCDEF  and  ABCDEF  and 
"gains"  0-vertex  ABCDEF;  increasing  c  loses  ABCDEF  and  "gains"  ABCDEF;  d  sym¬ 
metrically;  decreasing  e  loses  ABCDEF,  ABCDEF,  and  ABCDEF;  increasing  f  loses 
ABCDEF  and  "gains"  ABCDEF;  decreasing  T  "gains"  ABCDEF  and  ABCDEF.  Thus  a 
combination  of  changes  will  be  needed  to  effect  a  realization  by  iteration;  the  "hill" 
being  climbed  requires  a  more  subtle  approach.  This  is  strong  reason  for  considering 
more  direct  methods,  as  is  outlined  above. 

The  main  conclusion  of  this  section  is:  If  we  decide  in  a  recognition  problem  to 
try  to  use  a  threshold  decision  function,  then  switching  theoretic  methods  for  synthesiz¬ 
ing  the  function  are  preferable  to  the  conventional  Bayesian  approach. 


*  R.  L.  Mattson,  "  A  Self -Organizing  Binary  System, "  PROC  EJCC,  pp.  212-217, 
December  1959.  — 
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D.  CALCULATION  OF  MEASURES 


The  calculation  of  the  quantities  m  (f),  m  (f  x  j )  from  an  algebraic  representation  of 
f  can  be  made  very  easily,  providing  f  is  2-monotonic,  as  follows: 

1.  Put  the  representation  into  canonical  form  (page  67  of  thesis  [  W-4] );  call 
this  form  P.  (We  will  assume  P  is  positive,  but  nearly  the  same  procedures 
apply  in  the  general  case. ) 

2.  Write  the  numbers  2n  1  (decimal  form)  under  the  variables  x  i  ,  in  each 
appearance  as  the  last  factor  of  a  term  of  P.  Then  m  (f)  is  simply  the  sum 
of  these  numbers. 

Example:  (x^  is  A,  etc.) 

A  [  B  (C  +  D  +  EF)  +  C(D(E  +  F)  +  EF)]  +  BCDE 
8  4  1  2  11  2 

m(f)  =  8  +  4  +  1  +  2  +  1  +  1  +  2=19. 

Proof;  In  general, 

m  (f)  =  m(f  )  +  m(f-  ) 

X  X 

for  any  argument  x.  But  f  is  2-monotonic  by  assumption,  and  P,  being  in  canonical 
form,  is 


P  =  x1  Q  +  R 

where  Q  and  R  don't  involve  x  .  Thus 

m  (f)  =  m  (g  +  h)  +  m(h) 

where  g  is  represented  by  Q,  h  by  R.  But  by  theorem  18  (page  67)  of  thesis  [  W-4  ] , 
R  -  Q,  so  that  h  -  g;  i.  e. ,  g  +  h  =  g.  Then  J 

m(f)  =  m(g)  +  m  (h). 

This  simple  rule  of  decomposition,  applied  recursively,  gives  the  rule  stated  above. 

(END  OF  PROOF) 


37 


3.  To  compute  the  m  (fXj) .  below  each  numerical  entry  of  the  previous  step, 
except  the  l's,  write  one  half  of  that  enti-y. 

4.  Compute  m(fsi)  as  follows:  Moving  from  the  left  and  starting  on  the  bottom 
row,  add  up  entries  until  an  xj  is  encountered-  then  skip  over  whatever  the 
xi  might  multiply  into,  and  move  up  to  the  upper  row.  adding  entries  until 
an  x  j  with  j  £  i  is  encountered.  Then  move  bade  down  again,  and  continue 
as  before.  The  sum  will  be  m(f g j  );  then  m (f x  j  )  =  m  (f)  -  m(f  jj).  (It's 
easy  to  generate  a  slightly  more  complicated  rule  to  calculate  m(fXi  ) 
directly;  this  can  he  used  as  a  check.) 

Example: 

A  [  B  (C  +  D  +  EF)  +  C  (D  (E  +  F)  +  EF)]  +  BCDE 

8  4  1  2  11  2 

4  2  1  1 


8.  19  -  8  =  11. 
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VIII.  APPENDIX 


by  R.  0.  Winder 

The  83  SD  symmetry  types  for  four  arguments,  with  Chow  parameters 
and  realizations  by  3-input  majority  gates  (optimal  number  of  levels) 
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PART  2 


RELIABILITY  OP  SWITCHING  NETWORKS 


I.  INTRODUCTION 


This  part  of  Special  Scientific  Report  No.  2  also  consists  of  a 
series  of  sections;  Section  II  constitutes  a  survey  of  the  application 
of  redundancy  techniques  in  switching  systems  for  the  improvement  of 
reliability.  Section  III  discusses  several  extensions  of  earlier  work 
on  a  specific  scheme  (reported  in  Special  Scientific  Report  No.  I)  -- 
the  recursive  triangles,  and  Section  IV  introduces  a  new  line  of  explo¬ 
ration:  reliability  in  systems  with  memory  and  feedback. 

Section  II  is  an  outgrowth  of  the  Symposium  on  Redundancy  Tech¬ 
niques  for  Computing  Systems  which  took  place  in  Washington  last  year. 
It  is  intended  to  survey  the  field  with  the  dual  purpose  of  seeing 
what  is  being  done  outside  of  our  work  at  RCA  Laboratories  to  apply 
redundancy  to  increase  reliability  and  evaluating  our  recursive 
triangle  scheme  in  the  light  of  what  we  have  observed.  We  discuss 
those  methods  which  are  most  relevant  to  the  problems  that  we  ourselves 
are  considering.  (In  particular,  we  discuss  the  papers  presented 
at  the  Symposiun). 

Although  many  techniques  have  been  suggested  for  introducing 
redundancy  into  computing  systems,  only  those  which  require  very  small 
additional  quantities  of  equipment  have  gotten  beyond  the  paper  design 
stage.  Increased  computational  capability  is  a  very  easy  concept  to 
grasp,  but  the  concept  of  increased  reliability  is  not  well  understood. 
In  fact,the  former  is  regarded  as  a  goal  of  the  system  while  the  latter 
is  often  thought  of  as  an  obstacle  to  be  overcome.  Consequent ly? the 
two  have  been  separated  and  people  have  found  it  easy  to  justify  large 
additional  expense  for  the  former  and  far  more  difficult  to  justify  it 
for  the  latter.  The  two  are  not  at  all  independent  but  until  many  more 
people  appreciate  the  relation  between  them,  the  systems  proposed  to 
achieve  reliability  through  redundancy  will  be  slow  to  be  adopted.  A 
purpose  of  Section  II  is  to  describe  various  techniques  for  redundancy 
as  well  as  to  illustrate  where  each  technique  is  most  applicable. 


49 


Section  III  collects  and  presents  the  results  of  a  set  of  brief 
explorations  of  the  properties  of  recursive  triangular  networks.  It 
is  divided  into  three  distinctly  disjoint  subsections.  The  first 
establishes  that  in  the  case  of  the  three -input  rectifier  "and"  gate 
(which  we  studied  in  detail  in  Special  Scientific  Report  No.  1) 
triangular  recursion  can  be  used  to  improve  the  reliability  of  the 
gate  no  matter  how  unreliable  the  rectifiers  are.  The  second  sub¬ 
section  deals  with  attempts  to  make  use  of  the  recursive  triangle  to 
improve  the  reliability  of  a  nonsymmetric  Boolean  function  (ABvCD) . 

Both  rectifier  and  threshold  realizations  are  investigated.  This 
investigation  indicates  that  the  most  promising  level  at  which  to 
apply  recursive  triangulation  is  the  basic  "and"  gate,  "or"  gate  level; 
where  improvements  of  reliability  can  be  obtained  as  discussed  in 
Special  Scientific  Report  No.  1 .  Finally,  subsection  three  deals  with 
attempts  to  utilize  the  technique  to  improve  the  reliability  of  the 
"nor"  function.  Improvements  in  reliability  were  obtained,  similar  to 
those  reported  in  Special  Scientific  Report  No.  I.  Maximum  improvements 
are  obtained  with  three  "nor"  gates  feeding  an  "and"  gate.  However, 
when  the  probability  of  complete  failure  of  the  "nor"  gate  becomes  large, 
a  majority  gate  or  "or"  gate  will  give  better  improvements. 

In  Section  IV,  the  feasibility  of  introducing  self-repairing 
capability  to  a  logical  net  is  discussed.  In  complex  logical  systems 
having  millions  of  component  units,  as  also  in  Satellite  systems,  it  will 
be  very  desirable  to  have  a  logical  organization  capable  of  'self-repair'. 
Also,  one  would  like  to  have  a  net  take  preventive  measures  to  avoid  any 
failure  that  may  be  anticipated. 

A  logical  scheme  for  organizing  logical  nets  having  the  above 
capabilities  has  been  briefly  studied.  In  this  scheme  a  logical  net  will 
be  provided  with  the  facility  to  repeat  a  calculation  with  an  alternate 
net  (subnet)  whenever  an  error  is  detected.  Substitution  of  a  net  (sub¬ 
net)  may  also  take  place  at  the  anticipation  of  an  error  occurrence.  In 
such  a  scheme,  greater  reliability  of  operation  of  a  net  would  be  achieved 
at  the  cost  of  increased  computational  delay.  As  recalculation  will  occur 
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only  at  the  Incidence  of  an  error,  the  net  will  be  making  optimum  use  of 
the  computational  time.  Also,  as  the  network  deteriorates  due  to,  say, 
aging,  its  'reliability'  will  not  decrease,  —  instead  it  will  take  more 
time  to  do  a  computation. 

If  it  is  required  that  each  computation  done  by  the  net  should  take 
only  the  shortest  necessary  delay,  then  one  finds  that  this  substitution 
scheme  offers  the  only  possible  way  of  introducing  self-repairing  capabi¬ 
lity  to  the  logical  nets.  In  the  context  of  this  scheme  the  problems  that 
arise  in  the  organization  of  such  nets  are  pointed  out.  Also,  some  examples 
of  such  nets  are  discussed  to  illustrate  the  techniques  of  analysis. 
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II.  REDUNDANCY  TECHNIQUES  FOR  SWITCHING  NETWORKS 
by  S.  Y.  Levy 

This  report  is  an  outgrowth  of  the  Symposium  on  Redundancy  Techniques 
for  Computing  Systems  which  took  place  in  Washington  last  year.  It  is 
intended  to  survey  the  field  with  the  dual  purpose  of  seeing  what  is  being 
done  outside  of  our  work  at  RCA  Laboratories  to  apply  redundancy  to  increase 
reliability  and  evaluating  our  recursive  triangle  scheme  in  the  light  of 
what  we  have  observed.  We  discuss  those  methods  which  are  most  relevant  to 
the  problems  that  we  ourselves  are  considering.  (In  particular,  we  discuss 
the  papers  presented  at  the  Symposium). 

In  the  case  of  devices  built  of  few  components  or  at  least  easily 
accessible  components,  it  has  almost  always  been  the  policy  of  manufacturers 
to  increase  reliability  by  either  derating  components  or  using  more  reliable 
ones.  By  and  large,  improvements  in  reliability  have  come  about  through 
improvement  in  the  reliability  of  the  components  used,  as  shown  by  most 
people's  experience  with  their  home  radios,  for  example.  But  increased 
reliability  of  components  has  led  to  attempts  to  build  still  larger  more 
complex  systems  which  need  the  increased  component  reliability  merely  to 
maintain  the  old  levels  of  system  reliability  (you  might  recognize  this  as 
a  sort  of  "Parkinson's  law"  of  technology  which  says  that  advances  in  techno¬ 
logy  result  in  an  expansion  of  applications  which  require  the  new  technology) . 
In  addition,  many  applications  of  electronics  are  such  that  the  cost  of  error 
(or  failure)  is  high  enough  to  warrant  large  additional  expenditure  to  prevent 
it.  Applications  have  arisen  for  equipment  in  hostile  environment  (e.g., 
radiation  belts,  war,  etc.)  where  it  is  desirable  to  maintain  operation  in 
spite  of  the  certainty  that  parts  of  the  device  will  fall  —  or  the  device 
will  be  expected  to  operate  in  an  environment  where  repair  or  replacement  of 
parts  is  impossible  (e.g.,  an  unmanned  satellite).  At  the  other  extreme 
there  is  the  new  batch- fabricated  technology  where  large  quantities  of  rela¬ 
tively  Inexpensive  devices  are  made  simultaneously  and  Interconnected  in  a 
functioning  array  but  with  component  yields  far  less  than  that  acceptable 
from  individually  produced  devices.  Can  redundancy  be  applied  to  this  techno¬ 
logy  to  make  use  of  these  less-than-perfect  arrays  of  components?  At  any 
rate,  it  is  abundantly  clear  that  there  is  need  for  suitable  techniques  for 
the  efficient  application  of  redundancy. 


52 


The  Reliability  of  Coherent  Systems  by  Esary  and  Proschan  (Boeing 
Aircraft)^- is  a  general  discussion  of  the  analysis  of  the  behavior  of  contact 
type  networks  (e.g.,  relay  networks,  unipolar  transistor  networks).  An  n-con- 
tact  network  is  described  by  an  n-component  vector  --  each  component  being  an 
indicator  of  the  state  of  the  corresponding  network  component  (x^  -  1  indicates 
the  i**1  component  is  operating  properly,  *  0  indicates  the  component  is  not 
operating  properly).  An  overall  structure  function  4 (X)  is  defined  (where 
jf  m  Xj,  x^, ...,  xfl)  such  that  $  (X)  ■  1  if  the  overall  network  is  functioning 
properly,  ? (X)  =  0  if  the  network  has  failed.  The  property  coherence  is 
defined  by  three  conditions 

1)  $ (t)  -  1  where  1  -  1,  1,  ...,  1 

2)  4 (3)  *  0  where  0*0,  0,  ...,  0 

3)  For  all  i(x  >  y )  —  $  (X)  >  2  (y) 

The  paper  deals  only  with  coherent  nets  and  their  behavior  under  a  form  of 
iteration  which  involves  replacing  each  contact  by  a  replica  of  the  entire 
network.  To  investigate  the  behavior  under  this  type  of  recursion  they  define 
h(p)  as  the  probability  of  the  network  operating  properly  as  a  function  of  p,  the 
reliability  of  an  individual  component.  h(p)  is  a  so-called  "S-shaped"  curve. 


p 


If  each  component  is  associated  with  a  p  such  that  h(p)  >  p,  then  the 
iteration  results  in  a  staircase  effect  leading  to  arbitrarily  high 
reliability  (for  example,  in  the  diagram  h(p*)  >  p*;  h(h(p*))  >  h(p*)  etc.) 

If  h(p)  <  p  the  staircase  leads  to  0  reliability  in  much  the  same  way.  How¬ 
ever,  the  authors  fail  to  give  a  reasonable  characterization  of  coherence  in 
terms  of  what  sort  of  restriction  it  represents;  in  fact, they  do  not  give  a 
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•ingle  example  of  a  noncoherent  network.  Winder  has  pointed  out  a  simple 
example  of  a  noncoherent  structure  which  we  include  here. 


1 


+ 


x  y 

3  4 


f(x,y) 


f(x,y>  *  xy 

Suppose  relay  4  is  not  functioning  properly  and  is  stuck  closed 
while  relays  1,  2,  3,  are  operating  correctly. 

Then 

f(x,y)  -  xy  v  x  s  x  , 


SO 

f (1,1, 1,0)  -  o 

Suppose  relay  4  is  stuck  closed  and  relay  3  is  stuck  open,  while  relays 
1  and  2  are  operating  correctly. 

Then 

f (x,y)  -  xy  , 
so 

1(1, 1,0,0)  -  1 


i(1, 1,0,0)  >§(1,1, 1,0)  and  the  network  is  not  coherent. 

Note  that  the  structure  function  f (X)  is  a  function  from  binary  inputs  to  a 
binary  output  and  so  may  be  regarded  as  a  switching  function.  We  can  suggest 
the  following  characterization  of  a  coherent  network. 


Theorem:  A  network  is  coherent  if  and  only  if  the  switching  function 

£ (X)  (not  to  be  confused  with  f(Y)  the  function  computed  by  the  network) 
is  positive  in  all  and  not  identically  0  or  1 . 

Proof:  1)  Ass  tine  the  network  is  coherent.  Then  consider  any  X  for 

which  i  (X)  ■  1 .  Changing  any  xi  from  0  to  1  can  never  decrease  § (X) 
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because  the  network  is  coherent.  Therefore  and  the 

-»  xi 

function  1  (X)  is  positive  in  all  x^. 

2)  Assume  J  (X)  is  positive,  and  not  identically  0  or  1 . 

Then  for  all  x,  $_-»$  ,  this  means  increasing  x  can  never  decrease 

-t  1  xi  x  ^  .» 

1  (X) .  Since  for  at  least  one  value  of  X,  1  (X)  »  1,then$(l)  -  1;  and 

A  A 

since  for  at  least  one  value  of  X,  1  (X)  ■  0,  1(0)  «  0, the  network  is 
coherent. 

Corollary:  Irredundant  circuits  are  coherent.  The  structure 

function,  $  ,  of  an  irredundant  circuit  is  the  "and" 
function  1  (X)  ■  x^  •  *••  xfl. 

If  one  considers  the  mode  of  failure  as  failure  to  operate  (to  remain 
stuck  in  rest  position)  then  all  frontal  circuits  (where  relays  are  normally 
open)  are  coherent  circuits  because  a  failure  can  only  open  a  path  and  make 
it  less  likely  that  the  network  functions  properly. 

Though  the  authors'  interest  in  reliability  leads  them  to  attempt  an 

iterative  procedure  where  each  element  of  the  network  is  replaced  by  a  replica 

of  the  entire  network,  the  type  of  circuit  which  they  are  studying  yields  to 

an  entirely  different  analysis  than  does  the  recursive  triangle.  In  the 
2 

Moore-Shannon  paper  on  redundant  contact  networks  (which  incidently  devotes 
a  great  deal  of  time  to  the  study  of  these  S-shaped  curves)  there  is  an 
interesting  contrast  of  the  two  technologies,  the  mechanical  (relay  or 

contact  type)  and  the  electronic  (rectifier  or  transistor  type).  In  comparing 

3  2 

the  Von  Neumann  redundancy  scheme  to  their  own, Moore -Shannon  note  that  where 

in  an  electronic  system  the  logical  combining  ("and",  "or",  etc.)  is  subject 

to  error,  in  the  contact  system,  this  is  obtained  by  merely  making  appropriate 

connections.  On  the  other  hand,  in  a  contact  network,  the  introduction  of 

copies  of  an  input  is  subject  to  error  (by  requiring  additional  contacts)  and 

a  variable  is  copied  in  an  electronic  network  by  merely  making  appropriate 

connections. 

*  5.  ■  1  (X^  with  x  -  1 

i  1 

I_  *1(X)  with  x  ■  0 
i  1 
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Tolerable  Errors  of  Neuron*  for  Infallible  Seta  by  Blum,  Ernesto  and 
Verbeek  (MIT)^  ia  an  Interesting  paper  —  it  would  bave  been  better  if  they 
included  an  algorithm  for  the  design  of  such  error-free  networks  (if  indeed  the 
procedure  is  algorithmic)  or  at  least  a  little  more  detail  which  would  substan¬ 
tiate  and  explain  the  claims  made  in  the  paper.  The  networks  formed  consist  of 
n  +  1  neurons  for  an  n- input  function  arranged  as  shown  below: 


Fig.  2  Three -input  neural  net 

It  is  somewhat  similar  to  our  triangular  network  except  that  these 
nets  are  nonhomogeneous .  No  hint  is  given  at  a  physical  model  for  the 

type  of  network  they  discuss  but  the  error-correcting  properties  are  quite 
astounding.  The  nets  themselves  have  been  presented  before  and  as  such 
represent  nothing  new.  However, the  authors  make  the  interesting  conment  also 
made  by  several  others  at  the  conference  that  the  choice  of  a  simple  majority 
element  for  the  combining  or  output  gate  is  a  naive  one  to  be  made  only  in  the 
face  of  no  knowledge  about  the  reliability  behavior  of  the  components  to  be 
combined  (with  the  exception  of  a  few  special  cases),  and  that  knowledge  about 
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such  factors  as  the  nature  of  failures,  and  the  bias  (if  any)  of  errors  can 
lead  to  a  better  choice. 


“Codes  and  Coding  Circuitry  for  Automatic  Error  Correction  Within 
Digital  Systems"  by  W.  H.  Kautz  (SRIJ* is  one  of  the  better  papers  presented 
at  the  Symposium.  Before  discussing  it,  we  would  like  to  inject  some  general 
consnents  about  the  idea  of  coding  for  error  correction  in  digital  computers. 
There  are  two  distinct  ideas  represented  here.  One  is  the  coding  of  infor¬ 
mation  to  ensure  accurate  transmission  or  storage  much  the  same  as  in 
comnunication  theory  (and  the  problems  are,  in  fact,  the  same).  The  other  is 
the  encoding  of  information  to  ensure  accuracy  of  computation  or 
logical  operation.  The  former  involves  encoding  of  data  in  the  hope  that  the 
coded  form  will  resist  alteration  of  the  information  in  the  handling,  but  in 
the  latter  the  handling  is  expected  to  alter  the  information  and  the  problems 
are  quite  different.  It  is  almost  exclusively  the  former  with  which  Kautz 
deals.  The  latter  problem  is  discussed  in  Gore's^and  Winograd-Cowan* s  papers 
presented  at  the  Symposium  as  well  as  in  many  papers  not  given  at  the  Symposium 
and  will  be  analyzed  in  some  detail  later. 

The  coding  for  reliable  transmission  and  storage  depends  on  certain 
simple  properties  of  the  words.  Basically, the  idea  is  a  simple  one  —  enough 
redundant  (check)  bits  are  added  to  each  word  so  that  the  Hamning  distance 
(i.e.,  the  number  of  bits  in  which  two  words  differ)  between  any  two  words  is 
2k  +  1  where  k  is  the  largest  number  of  errors  which  it  is  desirable  to  correct 
for.  Then  a  received  word  is  decoded  as  the  word  closest  to  (least  Hamming 
distance  from)  the  word  received.  In  actual  practice  a  systematic  technique 
for  generating  code  words  is  used  in  order  to  facilitate  the  design  of  equip¬ 
ment  to  perform  the  task,  but  the  principle  remains  the  same.  The  coding  is 
generally  a  form  of  "block"  coding  and  is  encoded  and  decoded  by  a  set  of 
linear  operations  (i.e., described  in  terms  of  "exclusive  or"  and  its  comple¬ 
ment).  (If  the  words  were  to  be  logically  manipulated  certain  additional 
constraints  would  be  imposed  to  ensure  that  the  check  resulting  from  a  compu¬ 
tation  is  also  a  proper  check).  Kautz  is  concerned  with  techniques  for 
physically  realizing  these  codes  as  well  as  the  quantity  of  equipment  which 
is  actually  needed.  He  points  out  there  are  several  types  of  systematic 
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codes  available;  the  usual  group  codes,  the  low  density  codes  which  require 
greater  redundancy  of  information  (more  bits)  but  are  easier  to  encode 
and  decode,  Berger  codes  which  detect  completely  biased  errors  (all 
0*8  or  all  1 *s )  and  arithmetic  codes  suitable  for  checking  the  arithmetic 
operations.  These  latter  codes  generally  derive  from  number  theoretic  proper¬ 
ties  rather  than  from  the  logical  properties  which  produce  the  other  codes. 

g 

[Peterson  in  a  paper  on  checking  an  adder  demonstrates  that  the  only  type  of 
checking  possible  where  the  adder  and  the  checking  circuitry  are  independent 
is  some  form  of  remainder  mod  p  check”]. 

Although  it  is  possible  to  build  multiple  error-correcting  circuitry, 
the  expense  involved  is  usually  so  great  that  it  is  not  done  (with  the 
exception  of  special  cases  like  completely  biased  errors  or  bursts  of  errors 
in  a  set  of  adjacent  locations).  Kautz  points  out  that  "signal  redundancy 
suffers  from  certain  basic  limitations:  For  all  but  a  definable  minority 
of  logical  circuits,  there  exist  some  types  of  faults  whose  errors  no  amount 
of  signal  redundancy  can  correct"  (see  discussion  of  noisy  computation  below). 

He  then  goes  on  to  say,  "this  theoretical  limitation  need  not  overly  concern 
us,  however,  for  three  reasons.  First,  this  minority  includes  several  circuit 
operations  of  considerable  practical  importance,  such  as  simple  data  transfer, 
parity  checking,  and  linear  (pure  "exclusive  or")  logical  circuits 
generally.  Second  we  rarely  need  to  protect  a  circuit  against  all  possible 
faults,  but  only  a  selected  class  deemed  to  be  most  likely.  Third,  recently 
it  has  been  shown  that,  under  certain  reasonable  assumptions,  a  network 
can  be  made  arbitrarily  reliable  with  the  proper  combination  of  signal  and 
circuit  redundancies."  [Unfortunately,  we  have  not  seen  the  proof  of  this 
third  statement]. 

9 

Ouadded  Logic  by  J.  G.  Tryon  (Bell  Labs.)  presents  a  technique  for 
introducing  redundancy  into  a  logical  chain.  Essentially, it  involves  quadrupli¬ 
cating  the  number  of  gates  and  then  systematically  interconnecting  them  so  as 
to  mask  faults.  It  is  Tryon's  hope  to  mask  these  faults  as  close  to  their 
source  as  is  possible.  His  approach  is  a  qualitative  one  —  he  is  not  immediately 
concerned  with  the  question  "how  much  does  quadding  improve  reliability?"  — 

*  A  remainder  mod  p  check  is  performed  by  carrying  as  check  bits  the  remainder 
when  the  word  is  divided  by  p.  It  then  follows  that  the  sum  of  the  checks  of 
two  nunbers  is  the  check  of  the  sud  (similarly  the  difference,  product,  quotient 
in  a  fashion). 
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he  asks  instead  -  "If  the  output  of  this  gate  is  stuck  high,  how  can  I 
correct  for  it?"  If  there  are  two  stages  of  logic  beyond  the  faulty  gate, 
then  quadding  is  an  answer;  if  not,  quadding  is  not  an  answer. 

Quadding  is  applied  to  a  chain  (gating  or  timing)  rather  than  to  a 
single  logical  element.  A  quadded  chain  has  four  times  the  number  of  gates 
of  a  nonredundant  chain,  and  each  gate  in  the  quadded  chain  has  two  times 
the  fan-in  and  two  times  the  fan-out  of  a  nonredundant  gate.  The  termination 
of  a  quadded  chain  is  four  lines,  and  the  decision  with  regard  to  how  to 
interpret  these  lines  in  the  case  that  they  are  not  all  in  agreement  is  out¬ 
side  of  the  domain  of  quadding. 

In  his  report  Tryon  presents  an  algorithm  for  quadding  which  we  will 
partially  reproduce.  Tryon  describes  the  technique  for  the  quadding  of 
combinational  logic  built  of  "and"  gates,  "or"  gates,  and  "inverters"  by 
three  rules: 

Rule  1.  In  circuits  in  which  levels  of  "and"  and  "or"  alternate, 
wiring  patterns  must  be  chosen  so  that  no  signal  encounters  the  same  pattern 
twice  in  succession  as  it  goes  along. 

So,  for  example,  in  the  diagram  of  the  quadded  circuit  the  connection 
pattern  between  the  A  and  C  gates  is  different  from  that  between  the  C  and  E 
gates. 

Rule  2.  Whenever  an  "and"  unit  feeds  an  "and"  unit  (or  an  "or"  unit), 
all  wiring  patterns  feeding  the  first  unit  must  be  the  same  as  that  between 
the  first  and  second  units.  An  example  follows  on  the  next  page. 
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The  final  rule  includes  the  first  two  and  is: 

Rule  3.  As  any  signal  flows  from  the  input  of  any  "And"  or  "Or"  unit, 
along  any  path,  the  parity  of  the  number  of  changes  in  pairing  that  it  encounters 
must  be  equal  to  the  parity  of  the  sum  of  (a)  the  mmber  of  negations  and  (b) 
the  nimber  of  "and"/"or"  transitions. 

Incidentally, negation  is  handled  by  applying  De  Morgan’s  laws  to  move 
the  negations  to  the  inputs,  quadding, and  then  moving  the  negations  back  (making 
the  appropriate  changes.) 

Thus  to  quad; 
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We  also  present  an  example  of  a  quadded  circuit  which  should  give  the 
flavor  of  the  technique.  Figure  3  is  an  example  of  a  circuit  in  both  non- 
redundant  and  quadded  forms.  Consider  the  quadded  version.  If  as  shown  all 
the  M  inputs  are  1 's  and  all  the  P  inputs  are  0's,  then  the  output  of  gates 
Aj  -  should  properly  be  O's.  Suppose  A^  is  malfunctioning  in  such  a  way 
that  its  output  is  a  1  instead.  Then  at  the  next  level  of  logic  ("or"  gates 
Cj  -  C^)  this  incorrect  I  spreads  to  2  gates  Cj  and  (assuming  Bj  -  have 
output  0).  But  at  the  following  level  of  "and"  gates,  the  situation  is 
corrected  and  the  output  is  the  proper  "0"  on  all  4  lines.  It  is  fortunate 
that  the  "or"  gates  Cj  -  were  not  the  final  level  in  the  chain.  Suppose, 
on  the  other  hand,  that  all  the  M's  and  P's  are  l's  and  that  the  outputs  of 
the  gates  A^  -  A^  are  proper  1 's,  but  that  the  output  of  is  an  incorrect 
0  (while  Cj  -  put  out  l's);  then  the  output  of  E^  and  E^  will  be  0  while  the 
output  of  E2  and  E3  will  be  1.  The  required  action  is  unclear! 

This  demonstrates  that  quadding  requires  at  least  two  levels  beyond 
the  source  of  a  single  error  in  order  to  correct  for  it;  —  this  implies 
that  quadding  is  more  effective  as  the  chain  increases.  This  would,  in  most 


63 


technologies,  prove  a  serious  drawback  since  a)  a  chain  terminates  at  a 
storage  element  (flip-flop,  memory,  etc.),  and  b)  the  usual  tendency  in  design 
of  a  system  is  to  try  to  keep  the  chains  as  short  as  possible  to  avoid  deteri¬ 
oration  of  signal  and  increased  propagation  time. 

Tryon  has  managed  to  avoid  these  problems  since  he  quads  memory,  timing 
devices  and  even  flip-flops.  (The  latter  through  an  interconnected  "nor" 
gate  type  construction  of  the  flip-flops) 


Fig.  4  Tryon  flip-flop.  It  is  composed  of  combinational  elements 
that  may  be  quadded  in  the  usual  manner. 

The  fact  is  that  quadding  is  distinctly  oriented  to  a  particular  type  of 
equipment.  (There  is  nothing  wrong  with  this;  in  fact,  we  believe  that  the 
type  of  redundancy  to  be  used  should  be  designed  to  fit  the  particular 
technology.  We  state  it  only  because  we  believe  that  it  should  be  noted). 
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Tryon  goes  further  in  the  design  of  his  technique  than  anybody  else 

whose  work  we  have  read.  He  has  given  serious  thought  to  the  problem  of 

maintenance  of  a  redundant  system.  In  a  system  which  is  designed  to  mask 
faults,  it  is  harder  to  detect  faults.  Consequently,  it  is  more  difficult 
to  locate  failures,  and  so  a  redundant  machine  would  be  more  likely  to 
accumulate  them  than  would  a  nonredundant  machine  in  which  they  are  almost 
immediately  detected.  As  a  result  there  is  a  crossover  point  in  time  where 
if  both  a  nonredundant  and  a  redundant  machine  are  still  operating,  the 
redundant  machine  is  more  likely  to  fail.  B.  R.  Saltzberg,  a  co-worker  of 

Tryon's,  has  devised  a  technique  for  isolating  nonredundant  segments  of  a 

quadded  network  in  order  to  check  for  failed  parts.  This  technique  (as  an 
interesting  extra)  demonstrates  how  redundant  power  supplies  may  be  used  to 
protect  the  system  against  supply  failures  as  well  as  logical  failures. 

Quadding  as  a  technique  is  applicable  to  relatively  good  components. 

In  the  face  of  reasonably  high  probability  of  multiple  failure  in  a  quad,  it 
is  not  effective.  For  the  same  reasons  it  is  better  suited  to  a  situation 
where  regular  maintenance  will  not  allow  multiple  faults  to  accumulate.  How¬ 
ever,  it  does  offer  a  method  of  introducing  redundancy  at  a  relatively  reason¬ 
able  price  (about  eight  times  that  of  a  nonredundant  machine)  and  in  a 
substantially  uniform  way  throughout  the  machine. 

Adaptive  Vote  Takers  Improve  the  Use  of  Redundancy  by  W.  H.  Pierce 
(Westinghouse) presents  a  variation  on  the  majority  logic  scheme  of  Von 
Neumann.  In  this  technique  certain  of  the  replicated  organs  are  assumed  to 
be  more  likely  to  be  Incorrect  than  are  others,  and  so  less  credibility  is 
attached  to  their  decisions.  In  effect,  a  more  consistently  reliable  record 
is  rewarded  by  a  greater  say  in  the  final  decision-making  element,  and  as 
failures  occur  the  voting  weights  are  continually  adjusted. 

This  is  the  most  complex  system  for  introducing  redundancy  which  we 
have  seen.  Since  one  of  the  great  appeals  of  a  binary  system  is  its  relative 
immunity  to  error,  we  find  the  introduction  of  the  analog  weight  adjustment  a 
possible  weakness.  In  addition  we  do  not  see  the  need  for  adjusting  the  weights 
over  a  continuum  since  the  threshold  decision-making  device  will  recognize  only 
a  relatively  small  number  of  different  functions  of  the  inputs  —  perhaps 
adjusting  the  weights  discretely  would  help  the  reliability  of  this  involved 
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device.  Also, the  Pierce  scheme  is  not  intended  tc  improve  the  computation 
part  of  the  system,  but  is  directed  only  at  the  decision  maker.  It  intro¬ 
duces  an  additional  large  (unchecked)  system  which  appears  to  be  another 
likely  source  of  error.  Although  we  find  the  idea  interesting  and  the  analysis 
a  really  intricate  piece  of  work,  we  also  find  it  impossible  to  consider  it 
seriously  as  a  technique  for  increasing  reliability. 

W.  H.  Mann  (Wes tinghouse)  in  Restorative  Processes  for  Redundant 
Computing  Systems^  presents  some  very  good  solid  thoughts  on  redundancy.  The 
first  section  of  the  paper  discusses  modes  of  placing  restoring  organs  into  a 
system  at  desired  intervals.  He  discusses  the  various  effects  of  the  distri¬ 
bution  of  these  organs  on  the  system  reliability.  His  discussion  is  really 
really  outstanding  when  he  discusses  common  assumptions  about  components 
and  the  way  they  fail  and  how  these  assumptions  affect  reliability  analysis. 

The  most  interesting  of  these  comments  follows: 


Assumptions  and  their  effects 

1).  The  effects  of  highly  improbable  failure  modes  are  negligible. 


The  flaw  is  that  a  failure  which  is  improbable  at  the  circuit  level 
may  be  highly  probable  at  the  large  system  level.  For  example: 


Basic  nonredundant  system 
System  size 

Signal  processor  failure  probability 
Nonredundant  system  failure  probability 


1000  stages 

0.0005 

0.394 


All  failures  capable  of  restoration  -  Redundant  system 

Restoring  circuit  failure  probability  0.001 

-8 

Redundant  stage  failure  probability  3.36  x  10 

System  failure  probability  3.36  x  10  ^ 

Redundant  system  with  additional  0.1  percent  of  failures  not  capable  of 
restoration 

Signal  processor  failure  probability 
Restoring  checked  failure  probability 
System  failure  probability 
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0.0005005 
0.001001 
750.5  x  10 


2).  A  false  1  is  as  likely  as  a  false  0.  This  is  Che  type  of 
assumption  which  would  lead  to  a  simple  majority  decision-making  element. 

If,  for  example,  one  considers  a  model  like  ours  of  the  rectifier  "and"  gate 
where  the  open  diode  acts  as  a  1,  then  the  outputs  of  "and"  gates  are  such 
that  all  output  0's  are  correct  but  some  output  I's  are  incorrect.  If  this 
is  the  case  an  "and"  gate  is  the  best  choice  for  a  combining  element.  If 
the  probability  of  a  false  0  becomes  positive  the  choice  of  combining  func¬ 
tion  should  move  away  from  "and"  on  the  lattice  of  Boolean  functions  going 
close  to  majority  as  the  probability  of  0  and  1  get  closer  to  equal  and 
finally  towards  "or"  as  the  probability  of  0  gets  larger  than  that  of  1 . 

Information  Theory  and  Redundancy 

This  section  will  cover  work  reported  over  a  long  period  of  time  by 
many  people.  In  pre-information  theory  days,  the  standard  procedure  used  to 
transmit  binary  information  through  a  noisy  channel  was  to  repeat  the  trans¬ 
mitted  signal  2n+1  times.  Then  at  the  receiving  end  a  majority  vote  would  be 
taken  to  determine  the  nature  of  the  transmitted  signal.  If  n  +  1  or  more 
I's  were  received  the  signal  would  be  interpreted  as  a  1 ,  otherwise  the  signal 
would  be  interpreted  as  a  0.  This  is  a  simple  example  of  an  n-error-correcting 
code  (since  if  there  are  n  or  fewer  errors  the  signal  would  still  be  interpreted 
correctly).  This  obviously  gives  an  increase  in  reliability,  but  at  a  price 
—  a  decrease  in  the  rate  of  transmission  of  information.  [To  be  n-error- 
correcting  causes  a  decrease  in  transmission  rate  to  tin,es  that  of  the 

noncorrected  case].  The  probability  of  error  for  this  particular  system  is 
-k/R 

given  by  pe  ~  2  where  R  is  the  rate  of  transmission  of  information.  By 

decreasing  the  rate  (R),  pg  can  be  made  to  decrease,  but  to  make  pg  arbitrarily 
small  it  is  necessary  to  make  R  arbitrarily  small  as  well.  Information  theory 
provided  a  different  solution  to  the  problem  of  increasing  the  reliability  of 
transmission  of  information.  If  one  defines  a  quantity  C  (the  channel  capacity) 
as  a  maximum  rate  at  which  it  is  possible  to  obtain  mutual  information  from 
the  output  of  the  channel  about  its  input,  then  by  a  type  of  coding  called 
block  coding  it  becomes  possible  to  transmit  information  through  that  channel 
with  a  probability  of  error  given  by  pg  »  R  >  c  where  R  is  ®8®in  the  rate 
of  transmission.  [From  this  relation  we  see  that  C  is  also  the  maximum  rate 
at  which  it  is  possible  to  receive  information  over  the  channel  with  arbitrarily 
high  reliability].  To  block  code,  take  the  sequence  of  bits  to  be  trans¬ 
mitted,  break  it  into  units  k  bits  long,  and  encode  each  k  bit  sequence  into 
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an  n  bit  sequence  (n  >  k);  then  transmit  the  n  bit  sequence.  At  the  receiving 
end,  the  n  bit  sequence  distorted  by  noise  is  decoded  into  the  proper  k  bit 
sequence.  If  R,  which  equals  k/n,  is  kept  constant  and  less  than  C,  then  by 
increasing  both  k  and  n  the  error  probability  may  be  made  arbitrarily  small. 

But  all  the  results  we  have  presented  thus  far  hold  only  for  the  case 
that  the  information  coming  out  of  the  decoder  is  intended  to  be  the  same 
information  which  went  into  the  encoder.  But  what  of  the  case  where  the 
information  is  not  just  being  transmitted  or  stored,  but  is  being  logically 
combined  with  other  information  (al30  appropriately  coded);  can  block  coding 
provide  a  technique  fcr  obtaining  arbitrary  reliability  in  some  more  efficient 
manner  than  the  iterative  methods  mentioned  above?  Von  Neumann,  in  his  paper 
on  probabilistic  logics,  expressed  the  opinion  that  it  could,  but  just  how 
eluded  him.  Elias12  noted  that  the  techniques  of  replication  and  majority  vote, 
or  iteration  to  improve  reliability  are  much  like  the  pre- informat ion  theory 
techniques  for  reliable  signal  transmission.  If,  instead  of  building  a 
redundant  computer  operating  at  a  higher  reliability  level,  one  chooses  to  use 
the  redundant  elements  to  build  extra  computers  operating  at  the  old  reliability 
levels,  then  one  could  perform  more  (though  less  reliable)  computation.  So  in 
a  sense  the  result  is  a  trade-off  of  computation  rate  for  reliability.  Elias 
set  about  investigating  this  question,  by  considering  information  to  be  com¬ 
bined  according  to  any  of  the  16  binary  functions  of  2  binary  inputs.  In 
particular, he  studied  the  "and''  function  (and  then  discussed  its  generali¬ 
zation).  Since  many  of  the  later  papers  refer  specifically  to  his  results,  we 
shall  reproduce  his  ingenious  proof  here. 

k”andmcircuits 


Fig.  5  Multiple  "and"  computer 
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The  "and"  circuit  shown  above  takes  the  "and"  of  k  successive  n  digit  blocks 
of  coded  input,  using  k-1  n  digit  delays  and  k-1  noisy  "and"  gates.  We  call 

k 

the  input  sequence  consisting  of  k  ones  1  ,  and  the  sequence  consisting  of 

k-1  1 

k-j  ones  and  j  zeros  1  J0J  etc.  The  n  bit  block  which  is  the  coded  version 
of  s  will  be  denoted  by  T(s),  so  T(1  )  is  the  sequence  which  comes  out  of  the 
encoder  when  a  sequence  of  k  ones  is  put  in.  Consider  the  following  set  of 
n-digit  blocks  which  are  possible  results  of  this  computation  as  they  come 
out  of  the  last  "and"  circuit  before  decoding: 


T(1k) 

•  T(1k)  ... 

...  T(1k)  •  T(1k) 

T(1k) 

*  T(1k)  ... 

...  T(1k)  •  T(1k_1  01 ) 

T(1k) 

•  T(1k)  ... 

...  T(1k'V)‘  T(1k‘202) 

It  k-1  k  1  k-1  k 

T(1  )  •  T(1  '(T)  ...  TCI  o  ')  •  T(0k) 

Each  line  can  be  obtained  from  the  preceding  line  by  "anding"  with  one  new 

V 

factor,  since  the  "and"  of  T(1  )  with  itself  any  number  of  times  is  still 
T(1  ).  But  adding  a  new  factor  by  "anding"  can  only  strike  out  some  of  the 
ones  which  are  present  in  a  sequence.  All  of  the  n  digit  sequences  above 
must  be  decoded  differently,  since  they  all  represent  computations  having 
different  results  (the  proper  result  for  each  row  is  the  argument  of  the 
rightmost  factor).  In  going  from  one  line  to  the  next  we  eliminate  at  least 
one  one.  If  we  let  d^  be  the  number  of  ones  eliminated  when  passing  from  line 
i  to  line  i+1,  then,  since  there  are  at  most  n  ones  in  the  first  line  and 
at  least  no  ones  in  the  last  line, we  have 

_k 

h  di  <  di  > ' 

i*1 

It  follows  that  if  n  <  2k  some  of  the  d^  will  be  just  one  and  a  single 
error  in  the  output  can  cause  two  adjacent  lines  to  become  confused,  so  that 
the  decoder  will  print  out  the  wrong  answer  when  such  an  error  is  made,  if 
it  prints  out  the  right  answer  when  no  errors  are  made. 
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From  this  it  is  concluded  that: 


1)  It  is  not  possible  to  get  a  reliability  for  the  block  greater  than 
that  present  in  the  individual  devices  (reliability  here  being  probability  of 
correct  output)  until  the  rate  of  transmission  drops  from  unity(k  ■  n)  to  j 
(2k  ■  n)  and  even  then  there  are  single  errors  which  the  decoder  can  detect 
but  only  attempt  to  correct,  with  probability  y  of  guessing  wrong. 

2)  The  rate  must  drop  to  j  (3k  =  n)  before  all  single  errors  in  the 
computations  shown  above  can  be  corrected.  Notice  that  n  «  3k  has  a  familiar 
ring  and,  in  fact, if  we  must  go  to  that  much  redundancy  we  could  as  well  and 
certainly  as  easily  (from  the  point  of  view  of  decoding  and  encoding)  go  to 
simple  iteration  of  the  ic^  digit  of  each  block.  So  at  least  as  far  as  minimum 
distance  between  words  between  codewords  is  concerned,  no  block  code  (for  the 
computer  considered)  does  better  than  simple  iteration  of  the  input 

digits  [and  as  mentioned  earlier  this  leads  to  a  capacity  which  is  zero], 

Elias  considered  all  the  possible  binary  functions  of  2  inputs  Xj  and  X£. 
Of  these,  he  conjectured  that  the  only  nondegenerate  functions  to  which  coding 
could  be  applied  to  achieve  arbitrary  reliability  were  "exclusive  or"  and 
"equivalence".  But  "exclusive  or"  and  "equivalence"  and  any  combination  of 
the  degenerate  functions  of  2  variables  do  not  form  a  complete  set  of  propo¬ 
sitional  functions.  There  are  Boolean  functions  like  "and",  :  or"  which  can 
not  be  constructed  from  these  functions,  and  so,  in  all  likelihood, one  could 
not  construct  a  complete  general  purpose  computer  from  these  block-coded 
functions . 

13 

Peterson  and  Rabin  prove  the  same  results  as  Elias  under  somewhat  more 
relaxed  restrictions,  and  in  fact  they  are  able  to  verify  (algebraically) 

Elias'  conjecture.  Their  results  are  summarized  here; 

1)  For  single-error  detection  codes  which  consist  of  the  original 
information  with  a  check  symbol,  there  is  no  simpler  system  than  making  the 
check  symbols  duplicates  of  the  information  for  any  nontrivial  logical 
operations  except  "exclusive  or"  and  "equivalence"  both  of  which  may  be 
checked  with  "parity  digits". 
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2)  For  general  block  coding  with  the  output  decoding  one  —  one  in 
the  absence  of  errors,  and  the  ceded  blocks  processed  digit  by  digit,  for 
"and",  "or",  "exclusive-or",  "equivalence"  the  same  code  must  be  used  at 
the  output  as  at  the  inputs  (assuming  that  in  each  case  the  sequence  of  all 
zeros  codes  into  the  sequence  of  all  zeros).  For  the  other  six  nontrivial 
logical  operations  the  input  and  output  codes  are  closely  related.  For  all 
logical  operations  except  "exclusive  or"  and  "equivalence",  the  operation 
done  on  the  coded  blocks  must  be  the  same  as  the  operation  being  checked. 
For  "exclusive  or"  and  "equivalence",  each  digit  in  the  coded  blocks  is  a 
parity  check  on  some  subset  of  the  digits  of  the  uncoded  block.  In  other 
words, group  codes  and  only  group  codes  can  be  used  to  check  these  two 
operations. 


3)  For  the  same  restrictions  on  coding  as  on  (2),  and  for  all  non¬ 
trivial  logical  operations  except  "exclusive  or"  and  "equivalence",  there 
is  no  simpler  coding  system  with  a  specified  ability  to  detect  or  correct 
errors  than  a  system  in  which  the  coded  sequence  consists  of  a  number  of 
copies  of  the  uncoded  sequence. 


Relying  heavily  on  the  previous  results  of  Elias  and  Peterson-Rabin, 
Winogwcex tended  these  results  to  the  m  variable  case  (m  >  2).  His  results 

HI 

are  summarized  "Of  all  the  2^  Boolean  functions  of  m  variables,  only  2 
functions  are  linear  [and  hence  may  be  block-coded  to  advantage];  namely, 

k»+£  kixi 


all  functions  which  can  be  represented  as  f(x^. 


0m+1 


for  some  *  1  or  0.  Of  those  2  functions  only  2  are  explicit  functions 
of  all  m  variables,  namely 


n 

I  xi  and 

i“1 


where  £  and  +  stand 
for  "exclusive  or" 


Elias  also  pointed  out  that  if  we  relax  certain  restrictions  such 
as  insisting  that  the  decoding  at  the  output  be  one  —  one  in  the  absence 
of  noise  then  we  can  do  some  of  the  computation  in  the  error-free  encoder 
and  decoder  and  improve  our  reliability.  To  this  technique  Elias  applied 
the  rather  graphic  name,  "cheating". 
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Again  1C  may  be  said  Chat  this  does  not  mean  Chat  only  iteration 
serves  to  check  computer  operations.  If  extra  logical  information  is  availa¬ 
ble  about  the  operation  being  performed,  then  an  appropriate  check  may  often 
be  designed,  as,  for  example, a  remainder  modulo  p  check  on  an  arithmetic  unit. 

In  addition, if  one  thinks  in  terms  of  applying  the  checks  to  code  the  function 
as  well  as  the  inputs  and  in  addition  "owns  up"  to  the  possibility  of  error  in 
the  encoder  and  decoder  then  the  coding  may  result  in  improved  reliability 
(though  not  arbitrarily  high  reliability). 

Recursive  Triangular  Nets^* 

The  recursive  triangular  network  has  been  described  in  previous  reports. 

2 

It  is,  like  the  Shannon-Moore  procedure  an  iterative  technique  for  applying 
redundancy  and  consequently  the  quantity  of  equipment  used  grows  exponentially 
with  the  nunber  of  stages  of  recursion.  Very  generally, the  technique  of 
triangular  recursion  is  this: 


Fig.  6  Triangular  recursion  method 
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Where  the  basic  gate  (whose  reliability  is  to  be  improved)  is  the  0  level 
gate  the  1  level  gate  is  formed  by  a  triangle  of  n  (the  number  of  inputs  — 
in  Fig.  6  n  =  3)  0  level  gates  feeding  a  0  level  gate;  and  in  general, 
an  N  +  1  level  gate  is  formed  by  feeding  n  Nc  level  gates  into  an  Ncn  level 
gate.  The  manner  in  which  a  gate  fails  as  well  as  the  way  in  which  it 
operates  are  factors  which  determine  whether  or  not  triangular  recursion  can 
be  used  to  improve  the  reliability  of  a  gate.  Our  analysis  of  the  reliability 
of  a  gate  derives  from  the  idea  that  a  failed  gate  computes  a  logical  function 
other  than  the  one  for  which  it  is  designed.  This  other  logical  function  may 
be  a  degenerate  one  insensitive  to  some  of  the  input  variables  (or  even  all 
the  input  variables  as  in  the  case  of  a  gate  stuck  at  1  or  0),  or  it  may  just 
be  a  different  function  of  all  of  the  variables.  A  set  of  recurrence  relations 
relate  the  probabilities  of  occurrence  of  each  of  these  failure  functions  at 
a  stage  of  recursion  to  the  value  of  these  probabilities  at  the  previous  stage. 

In  earlier  reports  these  relations  have  been  produced  and  analyzed  for  various 
probability  distributions  over  failure  and  design  functions.  Under  certain 
restrictions  on  the  distributions,  it  is  possible  to  continue  recurring  until 
any  arbitrary  degree  of  reliability  is  achieved,  but  with  most  realistic 
distributions,  a  maximum  value  of  reliability  is  reached  after  just  a  few 
stages  of  recursion,  and  if  recursion  is  continued  beyond  the  maximum,  relia¬ 
bility  will  drop  off,  eventually  falling  to  zero.  In  the  case  of  the  simple 
rectifier  n-input  "and"  or  "or"  gate  the  probability  of  correct  functional 
operation  goes  from  (l-p)n  (where  p  is  the  probability  of  failure  of  a  rectifier) 
in  the  basic  rectifier  gate  to  very  close  to  l-pn  in  a  few  stages  of  recursion. 
This  would  mean,  for  example,  that  for  a  4-input  gate  with  a  probability  of 
rectifier  failure  of  .01  that  the  probability  of  failure  of  a  gate  decreases 
from  3.94  x  10"2  in  the  nonredundant  gate  to  10-8  in  the  redundant  gate  or 
by  a  factor  of  about  25000.  Of  course,  the  price  paid  for  this  would  be  very 
high  indeed  —  it  would  require  625  times  the  equipment  in  a  nonredundant 
gate.  If  an  improvement  factor  of  about  20000  would  suffice,  this  could  be 
obtained  at  a  cost  of  about  125  times  the  nonredundant  equipment.  These 
quantities  of  equipment  sound  forbidding.  They  are  (although  Von  Neumann 
spoke  of  numbers  on  the  order  of  1000  to  25000  in  his  multiplexing  technique). 
Even  if  the  economic  considerations  were  not  by  themselves  sufficient  to  bar 
such  a  system,  certainly  considerations  of  power  consumption  and  bulk  would 
exert  a  strong  negative  influence. 
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However, considerable  improvement  in  reliability  is  obtained  at  the 
early  stages  of  recursion  where  the  amount  of  equipment  required  is  not 
nearly  so  large  as  in  the  examples  cited  above.  The  table  below  gives  some 
examples  of  improvement  at  the  early  stages  of  recursion. 


Nunber 

N 

PN 

P0 

Equipment  ratio 

of 

Stage  of 

P( failure)  at 
Recursion 

P 

redundant 

Inputs 

P 

Recursion 

rN 

nonredundant 

1) 

3 

0.2 

0 

.488 

i 

1 

1 

.112 

4.4 

4 

2 

.011 

44 

16 

3 

.009 

54 

64 

2) 

3 

0.01 

0 

3  x  10"2 

1 

1 

1 

3  x  10‘4 

100 

4 

2 

3  x  10-9  10, 

000, 000 

16 

3) 

4 

0.3 

0 

.76 

1 

1 

1 

.2 

3.8 

5 

2 

.011 

7 

25 

3 

.009 

8.4 

125 

Notice  that  case  2, which  is  the  most  striking  of  the  3  cases,  represents 
a  gate  made  from  components  which  are  clearly  worse  than  anything  commercially 
acceptable  today  —  rectifiers  which  fail  1  out  of  every  hundred  times  they 
are  called  on  —  (and  cases  1  and  3  are  considerably  worse  than  that).  The 
point  made  here  is  that  the  triangular  recursive  redundancy  is  applicable  to 
really  poor  components  —  techniques  like,  for  example,  quadding  are  certainly 
more  reasonable  to  use  in  the  case  that  the  likelihood  of  multiple  error  is 
small  indeed,  but  in  the  case  where  the  components  are  really  bad  and  the 
occurrence  of  multiple  error  i3  not  unlikely,  then  only  the  recursive  triangle 
of  all  the  techniques  discussed  is  applicable. 
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In  summary,  the  recursive  triangle  requires  large  amounts  of  redundancy, 
but  it  is  able  to  make  extremely  efficient  use  of  the  "extra"  components.  If 
constructed  with  components  with  a  low  error  rate  recursive  triangular  nets 
might  be  used  to  build  equipment  with  an  extremely  long  lifetime  (e.g., for  use 
in  a  space  craft  taking  a  long  journey).  On  the  other  hand  the  recursive 
triangle  might  be  considered  as  applicable  to  a  situation  where  a  great  ni*nber 
of  components  can  be  produced  cheaply  (and  perhaps  not  too  reliably), but  it 
is  still  desired  to  construct  a  useful  system  from  them. 

The  present 

The  current'state  of  practice*  of  redundancy  techniques  as  applied  to 
computing  systems  is  yet  far  from  anything  discussed  in  this  report.  So  far 
as  I  can  gather,  people  design  conservatively  and  stock  spare  parts  (usually 
in  the  form  of  'plug-in  packages')  on  the  one  hand,  and  where  the  application 
(and  financial  capability)  warrant  it,  as,  for  example, in  satellites  or  defense 
systems  like  SAGE  ,  they  introduce  entire  standby  systems  to  be  operated  in 
parallel  or  as  they  are  required.  Redundancy  applied  at  levels  lower  than 
duplication  of  the  entire  system  is  usually  present  in  the  form  of  extra 
parity  bits  for  detection  of  errors  in  information  transfer  and  remainder- 
modulo  -some-p  checks  for  detection  of  error  in  arithmetic  computation,  with 
correction  of  the  errors  provided  by  repeating  any  operation  shown  to  be  in 
error . 

Increased  computation  capability  is  a  concept  very  easy  to  comprehend, 
but  the  concept  of  increased  reliability  is  not  so  well  understood.  In  fact, 
the  former  is  regarded  as  a  goal  of  the  system  while  the  latter  represents 
an  obstacle  (i.e.,  the  enemy).  Consequently  the  two  have  been  separated,and 
people  find  it  easy  to  justify  large  additional  expense  for  the  former  but 
far  more  difficult  to  justify  it  for  the  latter.  Of  course, these  two  are 
not  at  all  independent,  but  until  many  more  people  come  to  think  in  terms  of 
the  ways  in  which  they  are  related,  the  systems  which  we  have  discussed  in 
this  report  will  be  slow  to  be  adopted. 
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The  most  ambitious  that  anyone  has  been  getting  at  present  (to  the 
best  of  our  knowledge)  is  to  attempt  some  of  the  techniques  discussed  in 
Kautz  paper  which  do  not  require  large  additional  quantities  of  equip¬ 
ment.  If  faced  with  an  appropriate  technology  (and-or-not-quite  reliable) 
then  a  technique  similar  to  what  Tryon9  suggests  could  be  the  next  step. 
If  on  the  other  hand  the  technology  calls  for  making  use  of  very  cheap  but 
unreliable  components*  then  a  very  good  case  could  be  made  for  adopting 
something  like  the  recursive  triangle.  And  between  these  two  techniques, 
cases  are  to  be  made  for  many  of  the  systems  which  have  been  discussed. 
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III.  SOME  THOUGHTS  ON  RECURSIVE  TRIANGULAR  NETWORKS 
by  S.  Y.  Levy 

This  report  presents  the  results  of  a  set  of  brief  investigations 
of  the  properties  of  recursive  triangular  networks.  It  is  divided  into 
three  distinctly  disjoint  sections.  The  first  establishes  that  in  the  case 
of  the  three-input  rectifier  "and"  gate  (which  we  studied  in  detail  in 
Special  Scientific  Report  No.  1)  triangular  recursion  can  be  used  to  improve 
the  reliability  of  the  gate  no  matter  how  unreliable  the  rectifiers  are. 

The  second  section  deals  with  attempts  to  make  use  of  the  recursive  triangle 
to  improve  the  reliability  of  a  nonsymnetric  Boolean  function  (ABvCD) .  Both 
rectifier  and  threshold  realizations  are  investigated.  Finally,  section 
three  deals  with  attempts  to  utilize  the  technique  to  improve  the  reliability 
of  a  complete  function  "nor". 
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PART  I  THE  3  INPUT  RECTIFIER  GATE 


In  the  three  input  rectifier  gate  the  probability  of  correct  operation 
3 

p{ABC)  ■  S  =  (1-p)  where  p  is  the  probability  of  a  rectifier  malfunctioning. 
In  this  section  we  will  show  that  no  matter  what  value  p  assumes  in  the 
region  (0,1),  the  first  stage  of  recursion  is  always  more  reliable  than  the 
basic  gate  (S^  >  Sq). 

Using  the  notation  introduced  in  the  previous-  reports  p{T)  *  T, 

p ( A }  *  p{B}  *  p (C }  *  Q,  p[AB)  *  p(AC)  ■  p{BC]  *  R,  p{ABC)  *  S,  we  write  the 

recurrence  relation  S,(Q  ,  R  ,  S  ,  T  ). 

1  o'  o’  o’  o' 

S,  -  3Q  S  +  18R  3  +  3R  S  2  +  6R  S  T  +  18Q  R  2 

1  0  0  O  00  000  0  0 

+  18Q  R  S  +  18R2  S  +  6Q  2  S  +  24R  3  S  +  S  4 

o  o  o  o  o  o  o  o  o  o 

+  18R  2  S  T  +  3S  2  T  2  +  3S  3  T  +  63Q  2  R  S 

ooo  oo  oo  ooo 

+  63Q  R  2  S  +  27Q  2  S  2  +  9Q  S  3  +  27R  2  S  2 
o  o  o  o  o  o  o  o  o 

+  9R  S  3  +  18Q  R  S  T  +  18Q  S  2  T 
oo  oooo  oo  o 


+  18R  S  2 

T  +  54Q  R  S  2 

o  o 

ooo 

Substitute 

Q0  *  p20-p) 
ro  -  p(i-p)2 

SQ  «  (1-p)3 

T  »  p3 
o  v 

into  the  above  equation. 

*3 

Then  since  we  seek  the  region  where  S1  >  Sq  we  divide  Sj  by  Sq  =  (1-p)  . 

The  polynomial  which  results  is: 

1  +  4p  +  6p2  -  1 2p3  -  6p4  +  21 p5  -  66p6  +  108p7  -  72p8  +  71  p9. 

It  is  plotted  on  the  graph  where  it  appears  rather  clearly  that  this  ratio 
is  always  greater  than  1,  for  (0  <  p  <  1).  Since  the  improvement  which 
results  from  recursion  is  greater  as  the  nunber  of  inputs  increases  it  is 
expected  that  the  result  holds  for  all  n  >  3  (and  in  fact  Maitra  has  proved 
the  result  for  n  ■  2  as  well). 
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p 


Fig.  7  Improvement  ratio  at  first  level  of  recursion 
(3-input  rectifier  gate) 
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PART  II.  THE  FUNCTION  ABvCD 


Suppose  first  that  it  is  desired  to  synthesize  the  function  ABvCD 
from  a  network  of  rectifier  gates.  Suppose  further  it  is  desired  to  make 
use  of  triangular  recursion  to  improve  the  reliability  of  these  gates.  How 
should  the  recursion  be  applied?  There  are  two  obvious  alternatives. 

1)  Apply  triangular  recursion  to  a  basic  triplet  consisting  of  two  "and" 
gates  feeding  an  "or"  gate 


At)  CO 


LEVEL  l  RECURSION 


2)  Apply  triangular  recursion  to  each  of  the  gates  which  compose  the 
basic  triangle 


AB  CD 


Then  first  consider  Case  1.  Tabulate  the  behavior  of  a  zero  level  gate. 
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I 


Number  of  members  of 

Symmetry  type 

that  synmetry  type 

Nunber  of 

resulting  from 

which  can  occur  as  a 

failures 

p(n  failures} 

these  failures 

result  of  n  failures 

0 

O-P)6 

ABvCD 

1 

1 

P(l-P)5 

BvCD 

4 

AB 

2 

2 

I,.  %4 

P  (1-p) 

T 

2 

AvC 

4 

A 

4 

F 

1 

AB 

4 

3 

3/1  N3 

P  (1-P) 

T 

6 

F 

4 

A 

8 

AB 

2 

4 

4,,  \2 

P  (»*P) 

T 

5 

F 

6 

A 

4 

5 

p5(1-p) 

F 

4 

T 

2 

6 

6 

P 

F 

1 

The  probabilities  associated  with  the  different  operating  states  at  the 
basic  triangle  level  may  be  computed  easily  with  the  aid  of  this  table. 
Since  the  purpose  of  this  report  is  only  a  brief  inspection  of  this  problem 
we  will  assume  a  value  for  p,  and  compute  the  probabilities. 


p  “  0.1 

The  distribution  of  operating  states  in  the  basic  gate  is: 
p(ABvCD}  ■  (1-p)**  ■  .53 

p{AvCD}  -  p(BvCD)  *  p{ABvC)  ■  p{ABvD)  *  p(1-p)^  *  .059 
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.073 


p(AB}  -  p{CD)  -  pd-p)3  +  2p2(1-p)4  +  p3(1-p)3  - 

2  4 

p(AvC)  *  p (AvDj  *  p { BvC }  »  p(BvD)  «  p  (1-p)  *  .007 

p[A}  -  p[B)  -  p  (C)  -  p[D)  -  p2(1-p)4  +  2p3 ( 1 -p)3  +  pA(1-p)2  -  .008 

p(T}  -  2p2(1-p)4  +  6p3(t-p)3  +  5p4(1 -p)2  +  2p5 ( I -p)  -  .018 

p{F)  -  p2(1 -p)4  +  4p3(1-p)3  +  6p4 ( 1 -p)2  +  4p5(1-p)  +  p6  -  .010 

With  the  aid  of  the  following  tables  we  can  compute  the  recurrence 

relation  which  will  tell  us  the  probability  distribution  of  the  operating 
states  in  a  first  level  recursion.  Each  of  the  following  charts  represents 
the  resultant  function  of  the  first  level  triangular  gate  when  the  apex 
gate  is  a  particular  function  (fixed  for  each  chart)  and  the  inputs  of  this 
apex  function  are  specified  by  the  left  column  and  the  heading  of  each 
colunn.  [These  charts  are  read  like  the  mileage  charts  on  road  maps].  For 
example, if  the  apex  gate  is  performing  the  function  A'  B'  (see  chart  I)  and 
lines  A'  and  B'  are  the  function  ABvD  (the  next  to  the  last  row)  and  ABvC 
(the  third  column  from  the  right)  respectively,  then  at  their  intersection 
ABvCD  is  the  resulting  function.  Notice  also  that  charts  I  and  II  are  half 
shown.  Therefore,  when  totalling  combinations,  each  entry  with  the  exception 
of  those  along  the  main  diagonal  is  assumed  to  appear  twice. 

In  order  to  calculate  p{ABvCD),  it  is  first  necessary  to  compute  the 
probabilities  of  the  states  which  result  from  a  conjunction  of  2  lines 
feeding  the  apex  gate.  These  are  easily  taken  from  chart  I.  (Notation:  we 
will  use  the  name  of  the  function  to  stand  for  its  probability  --  thus  (AB) 
is  p(AB}). 

First,  we  will  calculate  probabilities  of  a  network  performing  a  particular 
function  when  the  apex  gate  is  performing  a  conjunction. 

These  probabilities  are  conditional  probabilities  -  actually 
(AB)  ■  p(AB  |  Apex  gate  is  conjunction)  and  similarly  for  the  other  probabilities. 

(ABvCD)  -  (ABvCD) [2[4(AvC)  +  4(AvCD)  +  T)  +  (ABvCD)]  +  4(AvCD) (BvCD)  -  .646 
(AB)  -  (CD)  -  2(AB) [ (T)  +  2(A)  +  4(AvC)  +  4(ABvC)  +  (ABvCD)] 

+  (AB)2  +  2(A) (B)  *  .126 
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(A(BvCD))  -  (B(AvCD) )  -  C(ABvD)  -  D(ABvC)  -  2[(A)(BvCD) 

+  (A) (ABvCD) ]  -  .0094 

(ABC)  -  (ABD)  -  (ACD)  -  (BCD)  -  2(AB)(C)  -  .0012 
(ABCD)  -  2(AB) (CD)  -  .011 
(F)  -  2F(1-F)  +  F2  -  .02 


The  next  step  in  our  calculation  is  to  compute  the  probability  of  the  func¬ 
tion  ABvCD  resulting  from  the  recursion.  As  before  we  will  enunerate  around 
the  apex  function. 


Apex  function 


Combinations 


yielding  ABvCD 


p{  ABvCD | Apex  function)’ 
p{Apex  function) 


A' 


(ABvCD) 


4(.008)(.53)  -  .017 


A'vC'  (ABvCD)2  +  2(AB) (CD)  +  2(ABvCD)[(F)  4(.007)(.464)  -  .013 

+  (CD) ] 

A'  B'  (ABvCD)2  +  2 (ABvCD) [ (T)  +  4(AvC)  2(.073)(.64)  -  .094 

+  4(AvCD)  ]  +  4(AvCD) (BvCD) 


A'vB*  C'  Note  that  an  asterisk  indicates  that 

the  member  results  from  conjunct 
tens  and  its  value  is  taken  from  the 
preceding  table 

4(AB)  (C(ABvCD)),  +  4 (ABvCD)  (ACD)  . 

+  4(AByCD)(A(B\klD))  +  (ABvgD)  (ABCD) 

+  (ABvCD)  (ABvJCD)  +  2(AB)(£») 

+  2 [ (ABvCD) (AB)  +  (AB^(ABOCD) ] 

+  (ABvCD) (F)  +  (£) (ABvCD)  4(. 059) (.630)  -  .149 


A'B'  vC*  D’  Here  all  terms  which  appear  result 

from  a  conjunction. 


4 (AB) (ABvCD)  +  2(F) (ABvCD)  +  (ABvCD)2 
+  2 (ABvCD) (F)  +  8 (AB) (C (ABvD) ) 

+  8 (ABC) (ABvCD)  +  8(A(BvCD))(ABvCD) 

+  2 (ABvCD) (ABCD)  +  8(A(BvCD))(C(ABvD)) . 

(.53)(.877)  -  .465 
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Therefore  p(ABvCD)  -  .017  +  .013  +  .094  +  .149  +  .465  -  .738  which  is 
greater  than  .53  which  is  the  p(ABvCD)  in  the  basic  gate.  But  this 
improvement  has  been  bought  at  a  cost  of  5  times  the  amount  of  equipment 
in  a  nonredundant  gate. 

If  we  consider  the  second  configuration  described  earlier  in  the 
report,  where  we  applied  recursion  directly  to  each  2-input  gate  which 
composed  the  basic  network,  then  one  level  of  recursion  (shown)  costs  only 
3  times  the  amount  of  equipment  in  a  basic  gate.  In  an  earlier  report 
(Maitra)  the  probability  of  successful  operation  of  a  first  level  triangle 
of  2-input  rectifier  gates  with  pfrectifier  failure)  «  0.1  has  been  calcu¬ 
lated.  It  is  0.940.  Successful  operation  of  the  function  ABvCD  requires 
that  all  three  triangles  operate  properly.  This  occurs  with  probability 
(.940)3  «  .830  which  is  greater  than  .738  and  has  been  obtained  at  a  cost 
only  3/5  as  great.  This  preliminary  study,  then,  appears  to  indicate  that 
the  simple  basic  gate  is  the  more  promising  level  at  which  to  apply  triangular 
recursion. 

The  Threshold  Logic  Realization 

In  addition  to  a  realization  of  the  function  ABvCD  using  rectifier 
gates  we  attempted  a  realization  using  threshold  gates.  As  in  our  previous 
report  (Special  Scientific  Report  No,  1)  it  was  assumed  that  errors  in  operation 
resulted  from  variation  in  the  bias. 

The  first  threshold  realization  we  considered  was: 


C  D 


In  this  case  recursion  yielded  no  stability  under  threshold  variation. 
The  upper  gate  can  be  made  more  reliable  by  recursion  but  the  lower  gate 
cannot.  So  we  attempted  to  use  the  same  network  as  before  to  synthesize 
the  function: 
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Again  our  assunptions  are  that  errors  in  operation  result  from 
variations  in  the  bias.  These  variations  are  represented  by  a  normally 
distributed  noise  signal  superimposed  on  the  bias  signal  and  centered  at 
the  appropriate  level.  Equally  weighted  two  input  threshold  gates  can 
realize  only  4  positive  Boolean  functions:  the  constant  T  (or  1),  Disjunction 
(or).  Conjunction  (and),  the  constant  F  (or  0). 

The  tables  for  the  resultant  function  again  taken  for  each  apex 
function  follow 


A,  B 

T 

D 

C 

F 

T 

T 

T 

T 

T 

D 

T 

AvBvCvD 

AvBvCD 

AvB 

Apex  function 

C 

T 

ABvCvD 

ABvCD 

AB 

Disjunction 

F 

T 

CvD 

CD 

F 

A,  B 

C,  D  _ 

>  T 

D 

C 

F 

T 

T 

CvD 

CD 

F 

D 

AvB 

(AvB) (CvD) 

(AvB) CD 

F 

Apex  function 

C 

ab 

(AB) (CvD) 

ABCD 

F 

Conjunction 

F 

F 

F 

F 

F 

The  probabilities  for  the  basic  gate  can  be  computed  from  the  tables.  The 
notation  used  below  istM^  is  the  probability  that  a  gate  which  is  intended 
to  be  an  N  gate  is  an  M  gate; thus,  Cp  is  the  probability  that  a  disjunction 
(or)  gate  will  act  as  a  conjunction  (and)  gate 

p{T)  -  Td+Cd[Tc2]  +Dd[2Tc  -  Tc2] 

P(F)  «Fd+Cd[2Fc  -  Fc2)  +Dd[Fc2] 
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p(ABvCD)  -  DdCc2 
p{AB]  *  p£CD}  -  DdCcFc  +  CdCcTc 
P (AvB }  -  p{CvD}  -  DdDcFc  +  CDDcTC 
p(ABvCvD)  *  p [AvBvCD)  *  D^C^D^, 

p(AB(CvD) )  -  p[(AvB)CD)  -  D^pOp 
2 

p(AvBvCvD)  “  DdDc 
p(ABCD)  -  CDCC2 
p(ACvADvBCvBD)  -  D^2 

If  we  consider  o ■  ■  0.50  in  our  normal  distribution, then 

Cc  -  DD  -  .6826900 

D  «  C  -  .1573050 

C  D 

Tp  -  Fc  -  .1586550 

Tc  -  Fp  -  .0013500 

Inserting  these  values  into  the  equations  on  the  previous  page,  we  get: 

p{T}  -  .1604961 

p(F}  -  .06052949 

p(ABvCD)  -  .3181783 

p(AB)  -  p{CD}  -  .0741341 

p(AvB)  -  p{CvD]  -  .0170767 

p(ABvCvD}  -  p (AvBvCD)  -  .073314 

p(AB(CvD) }  -  p( (AvB)CD]  -  .01689 

p{ABCD)  -  .073314 

p(AvBvCvD)  ■  .01689 

p{ (AvB) (CvD) )  «  .01689 

We  are  recurring  the  entire  network  as  in  the  first  case  and  we  shall 
develop  our  expression  around  the  functions  of  the  apex  gate.  Again, we 
make  use  of  charts  to  assist  our  calculation  and  we  use  the  sane  abbreviations 
as  before  (x)  s  p(x). 
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ABCvABDvCO  F  vCD  AB(CvO)  CD  vACDvBCD  VCD  vCD  vCD  AB(CvD)  (AvB)CD  A  BCD 

ACvADvBC  ABCvABD  ACvADvBC  ACvADvBC  ACvADvBC 

vBDvCD  F  vCD  AB(CvD)  CD  (AvB)(CvD)  vBDvCD  vBDvCD  vBDvCD  AB(CvD)  (AvB)CD  A BCD 

ABCvABD  ABCvABD  ABCvABD  ABCvABD  ABCvABD  ABCvABD 

vACDvBCD  F  vACDvBCD  AB(CvD)  (AvB)CD  vACDvBCD  vACDvBCD  vACDvBCD  vACDvBCD  AB(CvD)  (AvB)CD  A  BCD 
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licion* 


Combinations  Yielding  ABvCD 


Apex  gate 
2(AB) 

2 (AvB) 

ABvCD 

ABvCvD 

AB(CvD) 

ABCD 

(AvB) (CvD) 


2  (ABvCD)  T  +  2  (ABvCD)  (AvBvCD)  +  2  (ABvCD)  (ABvCvD) 

+  2 (ABvCD) (AvBvCvD)  +  (ABvCD)2 

This  expression  (above)  will  be  denoted  by  Q  and  the  corresponding 
expression  when  the  apex  function  is  AvB  will  be  denoted  by  0. 

In  addition  the  probability  of  a  function  F  resulting  from  either 

an  AB  apex  or  an  AvB  apex  will  be  denoted  by  (F)  or  (F)Q 

a  (3 

respectively. 

2 (ABvCD) (F)  +  2(AB) (CD)  +  2(ABvCD)(AB)  +  2(ABvCD)(CD) 

+  (ABvCD)2  +  2 (ABvCD) (ABCD)  +  2(ABCD)(AB(CvD))  +  2  (ABvCD) ((AvB) CD) 

2(a)(FQ)  +  2(ABa)(CDa)  +  2(a)(ABa)+2(a)(CDQ)  +  (a)2  +  2(a)(ABCDQ) 

+  2(a)([AB(CvD)]a)  +  2  (a)  ( [(AvB)  CD  y  +  2  (ABvACvADvBCvBDvCD)a 

(a)(Fp)  +  (a)  (0)  +  (a)(ABp)  +  (a)(CDp)  +  (^(F^  +  (^(AB^ 

+  (P)(CDa)  +  (CDa)(ABp)  +  ([AB(CvD)]a)(p)  +  ([AvB)CD]a)(p) 

+  (CDp)(ABa)  +  (a)([AB(CvD)]p)  +  (a)([  (AvB)CD]p)  +(ABCDa)0) 

+  ([ABvACDvBCD]  )(0)  +  ( [ABvACDvBCD]  ) (CD  )  +  (ABCDa)(a) 

CZ  CL  p  p 

+  (a) ( [ABvACDvBCD ]p)  +  ([ ABvACDvBCD 1^) (CDQ)  +  ([ABCvABDvCD]a)(0) 

+  ([ABCvABDvCDL)(AB0)  +  ([ABCvABDvCDl  )(a)  +  ([ABCvABDvCD]_)(AB  ) 

c*  p  p  p  U 

+([ ABCvABDv CD ]Q) ( [ABvACDvBCD]^)  +  (a) ( [ABCvABDvACDvBCD]^) 

+  ( [ABCvABDvCD]^) ( [ABvACDvBCD ]p)  +  (  0)  (  [ ABCvABDvACDvBCD  ]  ^ 

(1^(0)  +  (a)(0)  +  ([  ABvCvD ]Q)  ((3)  +  (a)  (Tp)  +  (a)  ([  ABvCvD  ]p) 

+  ([AvBvCD]^  (0)  +  ([  AvBvCvD  ]a)(p)  +  ([AvBvCD]p)  +  (a)  ( [AvBvCvD  ]Q) 

+  ([ABvACvADvBCvBDvCDl  )0)  +  (a)  ([ABvACvADvBCvBDvCDL) 

a  p 

2(a)  (Ta)  +  (a)2  +  2(  [ABvCvD  ]a)(a)  +  2  ( [AvBvCD  ]a)(d) 

+  2(  [AvBvCvD  y  (a)  +  2([ABvACvADvBCvBDvCD]a)(a) 

20)(Tp)  +  02  +  2([ABvCD]p)(0)  +  2([AvBvCD]p)  (p)  +  2([AvBvCvD]p)(0) 

+  2([ABvACvADvBCvBDvCD]_)(0) 

P 
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(AvB)v(CvD)  20)(Fp)  +  (p)2  +  2(p)(ABp)  +  2(P)(CDp)  +  2(ABp)(CDp) 

+  2([AB(CvD) ]p) (P)  +  2([AvB)CD]p)(p)  +  2(ABCDp)(p) 

+  2([ABvACDvBCD]p)(p)  +  2([ABvACDvBCD]p) (CDp)  +  2([ABCvABDvCD]p) (p) 
+  2([ABCvABDvCD]p)(ABp)  +  2([ABCvABDvCD]p)([ABvACDvBCD]p) 

+  2(P)([ABCvABDvACDvBCD]p) 


F  -  2F-F2 
(AB)a  -  <CD)a  = 


2(AB) (ABvCD)  +  (AB)2  +  2(AB)(AvB)  +  2(AB)(T) 
+  2 (AB) (AvBvCD)  +  2(AB) (AvBvCvD) 


(ABCD)a 


2(ABCD)(1-F)  -  (ABCD)2 


[AB(CvD)  ]Q  - 
[(AvB)CD]a  - 


2(AB(CvD))(T)  +  2 (AB(CvD) (ABvCD)  +  2(AB)(CvD) 

+  2(AB) (AB(CvD))  +  2 (AB) ((AvB) (CvD))  +  2((AB)(CvD))(AvB) 
+  2(AB(CvD))(CvD)  +  2(AB(CvD))(ABvCvD) 

+  2 (AB(CvD) ) (AvBvCD)  +  [AB(CvD)]2 


[ ABvACvADvBCvBDvCD  ]Q 

[ABvACDvBCD]q  - 
[ ABCvABDvCD  ]Q  * 

[ ABCvABDvACDvBCD  ]q  - 

Ta  " 

[AvBvCvD ]q  • 

[ABvCvD]a  - 
[AvBvCD ]  ■ 

TP  * 

FP  “ 

[AB(CvD) ]p  - 
[ (AvB)CD]p  - 


-  2 (AvBvCD) (ABvCvD) 

2 (AvB) (ABvCD) 

2 ( (AvB) (CvD) ) (ABvCD) 

T2 

2 

2 (AvBvCvD) (T)  +  (AvBvCvD) 

2 (AvBvCD) (T)  +  (AvBvCD)2 

2T  -  T2 
F2 

2(AB) (F)  +  (AB)2  +  2 (AB) (AB(CvD) )  +  2(AB)(ABCD) 
2[ (AB) (CvD) ] (F)  +  (AB(CvD))2  +  2 (AB (CvD)) (ABCD) 
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[ABCD]p  - 

[  ABvACDvBCD  ]  - 

P 

[ABCvABDvCD]  - 

P 


2(ABCD)F  +  (ABCD)2 


2 (AB(CvD) ) (CD) 


[ABCvABDvACDvBCDL  -  2 ( (AvB) (CD) ) ( (AB) (CvD) ) 

P 

[AvBvCvD ]Q  «  2 (AvBvCvD) ( 1 -T)  -  (AvBvCvD)2  +  2(AvB)(CvD) 

P 

+  2 (AvB) (ABvCvD)  +  2(AvBvCD)(CvD)  +  2 (ABvCvD) (AvBvCD) 


[ ABvCvD ]Q  - 

P 

[AvBvCD ]Q  -  2 (ABvCvD) (F)  +  2 (ABvCD) (CvD)  +  2 (ABvCD) (ABvCvD) 

P 

+  2(AB)  (CvD)  +  2  (ABvCvD)  (AB)  +  2  (ABvCvD)  (CD) 

+  2 (ABvCvD) (CvD)  +  (ABvCvD)2  +  2 (ABvCvD) (AB(CvD) ) 
+  2 (ABvCvD) (AvB) CD)  +  2 (ABvCvD) (ABCD) 

+  2 (ABvCvD) ((AvB) (CvD)) 


[ABvACvADvBCvBDvCD  L  -  2 ((AvB) (CvD)] [ABvCD] 
P 


Then  plugging  Che  values  into  these  expressions  get: 


Fa 

.11739516 

a  -  .30742597 

[ ABvACvADvBCvBDvCD ] 

.01074988 

(AB)a  -  (CD)Q 

.09237438 

(ABCD)a 

.1323777 

(ABvACDvBCD)  - 

(ABCvABDvCD)q 

.0471757 

(ABCvABDvACDvBCD)q 

.01086687 

(AvBvCvD)q 

.0057058 

(AvBvCD) a  -  (ABvCvD)a 

.0057058 

T 

a 

.025758998 

[AB(CvD)]a  -  ((AvB)CD]a 

.03095794 

.0036638 

3  -  .3132488 

(ABvACvADvBCvBDvCD) 

.01074806 

(AB)p  -  (CD)g 

.0278448 

(ABCD)p 

.0142649 
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(ABvACDvBCD)^  -  (ABCvABDvCD) ^ 
(ABCvABDvACDvBCD)p 

(AvBvCvD). 

P 

(AvBvCD)  *  (ABvCvD) 

P  P 

Te 

[AB(CvD) ]p  -  [ (AvB)CDjp 


.0025042 

.01074988 

.04441408 

.1164174 

.2952332 

.0048 

which  is  less  than  .318,  the 


and  finally  p{ABvCD]  «  .245 
probability  of  correct  operation  of  the  nonredundant  gate. 


In  addition  the  usual  recursion  of  the  basic  gate  yields  no  improvement 
in  reliability,  and  it  appears  that  in  this  case  triangular  recursion  is  of 
no  value  for  improving  the  operation. 
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FART  III.  THE  NOR  GATE 


Up  to  this  point,  we  have  not  actually  applied  recursive  triangulation 
to  a  complete  set  of  Boolean  functions  -  that  is  a  set  from  which  any  Boolean 
function  can  be  synthesized.  The  "nor"  gate  is  such  a  function.  Consider 
a  rectifier  transistor  realization: 


Fig.  8  Nor  gate 


If  we  consider  only  the  active  components  failing  then  the  distribution 
of  functional  states  resulting  from  the  component  failures  are  A,  "V,  7!,  XI, 

AC,  BC,  ABC,  T,  F.  These  are  the  same  as  in  the  case  of  the  3 -input  rectifier 
"and"  gate  except  for  the  addition  of  the  state  F.  If  we  apply  regular  re¬ 
cursion  then  1  level  of  recursion  can  not  yield  the  "nor"  function  and, in 
fact, from  one  level  on  the  function  computed  will  be  positive.  So  we  seek 
the  best  combining  element  for  "nor"  gates.  We  know  from  our  studies  of  the 
rectifier  gates  that  the  "and"  gate  would  be  the  ideal  combining  element 
were  it  not  for  the  presence  of  the  F.  So  we  will  study  the  effect  of  the 
F  if  we  attempt  to  combine  three  "nor"  gates  into  a  first  level  recursive 
triangle. 

We  assume  mixed  technologies  (more  than  one  type  of  gate  available) . 

The  object  of  the  study  is  to  determine  how  we  could  best  Improve  the 
reliability  of  the  "nor"  gate.  The  technique  used  compares  3  combining 
elements  --  a  rectifier  "or"  gate,  a  rectifier  "and"  gate,  and  a  majority 
gate. 
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A.  Rectifier  "or"  Gate  Combining  Element 


ABC  ABC  ABC 


The  distribution  on  the  nor  gate  is  again  over  A,  C,  AB,  25,  BC,  ABC,  F, 

T,  p{F }  ■  ,  using  a  rectifier  "nor"  gate  as  shown^with  p  *  0.2, and  using 

10 

the  notation  of  the  previous  report: 

Q  -  p{A}  ■  p{B)  -  p{C)  -  p2(1-p)(1^£)  -  (.04)(.8)(.98)  *  .0314 

.  JO 

R  -  p{AB)  -  p{AC}  -  p(BC)  -  p(1-p)  (1z£)  -  (.2)(.64)(.98)  -  .125 

3  10 

T  -  p{T}  -  (p  )(1^)  -  (.08) (.98)  -  .00785 

10 

S  »  p(ABC)  -  (1-p)3  (1^£)  -  (.51 2)(.98)  *  .504 

10 

p{F)  -  .02 

The  only  way  to  get  25?  from  this  combination  is  to  have  at  least  one 
of  the  input  gates  functioning  properly  and  the  diode  fed  by  that  gate  also 
functioning  properly. 

p{ABC)  -  3(p2) (1 -p)  (.504)  +  3p(1-p)2  ((.504)2  +  2(.504)(.02)) 

+  (1-P)3  ((.504)  +  3(.504)2  (.02)  +  3(.504)(.02)2) 

-  3(.032)(.504)  +  3(.128)((.504)2  +  (.504)(.04)) 

+  .512  ((.504)3  +  (.06)(.504)2  +  3(504)(.02)2) 

-  .0484 

p{ABC)  ■  .227  no  improvement;  in  fact,a  decline. 
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I 


Notice  that  if  the  combining  gate  is  improved  by  triangular  recursion  to 
its  maximum  reliability  p{or}  *  .991  that  the  probabilities  of  sensitivity 
to  only  1  or  only  2  inputs  goes  effectively  to  0  and  the  only  term  which 
need  be  considered  is  the  final  term. 

p{Slc)  -  (.991)(.144)  -  .143,  still  more  of  a  decline. 

In  fact,  a  comparison  of  this  calculation  with  the  last  shows  that 
greater  improvement  results  when  the  combining  element  is  not  operating  as 
an  "or"  gate  but  rather  when  it  is  malfunctioning. 

B.  "And"  Gate  Combining  Element 

The  recurrence  relation  is  almost  identical  with  the  relation  es¬ 
tablished  for  the  "and"  gate  except  that  a  factor  should  be  added  to  ensure 
that  no  input  function  is  F. 

p{Nor)  >  p{first  level  and  gate  recursion}  •  p{no  input  F} 

■  (.889) (.98) 3  (.889) (.94)  ■  .837  an  improvement. 

In  addition,  suppose  that  the  apex  gate  is  improved  by  triangular  recursion 
to  be  most  reliable.  Again  p(AB),p(A)  etc.,  go  to  0,  so  that  the  only 
combinations  left  to  yield  "nor"  are  the  following 

[S]  [6Q3  +  24 R3  +  S3  +  18QR2  +  3T2S  +  3TS2  +  63Q2R 
Apex 

+  63QR2  +  27Q2  +  3QS2  +  27R2S  +  9RS2  +  18QRT 
+  18QST  +  18RST  +  54QRS] 

S  -  .991 

Apex 

Then,  substituting: 

[.991 ][6(.0314)3  +  24( . 1 25)3  +  (.504)3  +  18(.0314)(.125)2 

+  3(.504)(.0785)2  +  3(.504)2  (.0785)  +  63(.0314)2  (.125) 

+  63(.0314)(.125)2  +  27(.0314)2  (.504)  +  9(.0314)(.504)2 
+  27 ( . 1 25) 2  (.504)  +  9(. 1 25 ) ( . 504) 2  +  1 8 ( . 03 1 4) ( . 1 25) ( . 0785 ) 

+  18( .0314) ( .504) ( .0785)  +  18(.125)(.504)(.0785) 

+  54( .0314) ( . 1 25) ( .504) ]  ■  .975,  a  decided  improvement. 
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C.  Majority  Gate  Apex  Gate 


ABC  ABC  ABC 


Since  we  are  trying  to  demonstrate  that  the  "and"  gate  is  the 
superior  combining  element,  we  can  only  emphasize  the  point  if  we  assume 
that  we  are  working  with  a  perfect  majority  gate.  Now  using  the  tables 
on  the  next  two  pages  and  expanding  around  one  of  the  3  input  gates: 


p{ABC)  -  F[6QR  +  6R2  +  2ST  +  S2  +  2S(1-F-S)] 

+  T[S2  +  2SF] 

+  3Q[S2  +  2RS  +  2FS  +  2RF] 

+  3R[S2  +  4RS  +  2QF  +  4RF  +  2R2  +  2SF] 

+  S[S2  +  2S(1-S)  +  2F(1-F)  -  2SF  +  2TS  +  6R2  +  6QR] 
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A 

c 

15 

A  C  BC 

ABC 

T 

T 

T 

T 

T 

T  T 

T 

T 

F 

A 

B 

C 

A3 

AC  BC 

15c 

T  F 

This 

chart 

and  the 

one 

following  are  for  the 

case  that 

the  apex 

is  a  perfect  majority  gate.  The  two  dimensions  of  each  chart  stand  for  two 
of  the  three  input  gates.  The  label  on  each  section  stands  for  the  third 
input  gate. 
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[.02] [6 (.03 14) (.125)  +  6(.125)2  +  2(.504)(. 00785)  +  (.504)2 

+  2(.504)(.476) ] 

+  [. 00785 ][(.504)2  +  2(.504)(.02) ] 

+  [3( .0314) ] [ ( . 504)2  +  2(.125)(.504)  +  2(.02)(.504)  +  2(.125)(.02)] 

+  [3(.125)][(.504)2  +  4(.125)(.504)  +  2(.0314)(.504)  +  4(.125)(.02) 

+  2(.125)2  +  2(.504)(.02)] 

+  (.504)[(.504)2  +  2(.504)(.496)  +  2(.02)(.98)  +  2(.00/85)(.504) 

+  6 ( . 1 25)2  +  6 ( .03 14) (.1 25) ] 

-  .74355 

This  represents  an  improvement,  but  not  nearly  as  great  an  improvement  as 
with  the  "and"  gate. 

Suppose  that  we  assume  a  much  higher  probability  of  F, occurring  say  at 
p{F)  -  .2 

Then  Q  -  .02>6 

R  -  .1024 

S  -  .4096 
T  -  .0064 

F  -  .2 

In  the  case  of  an  "or"  combining  element,  p(ABC)  -  .2490  (>.227,  the  previous 
value),  but  still  not  an  improvement  over  the  basic  gate. 

In  the  case  of  an  "and"  combining  element^  p{ ABC)  ■  (.89)(.8)3  ■  .455, 
a  decline  over  the  previous  value  (.837)  and  only  a  small  improvement  over 
the  basic  gate  reliability.  So  the  effect  of  the  "and"  combining  element 
appears  to  be  quite  sensitive  to  the  value  of  p{F). 

Finally,  the  majority  gate  combining  element  yields  p{ABC)  *  .675, 
which  represents  a  decline  in  the  previous  value  but  not  as  great  as  the 
decline  which  resulted  with  the  "and"  gate  combining  element.  If  instead 
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of  the  perfect  majority  gate  we  had  assumed  a  majority  gate  similar  to  the 
type  discussed  under  B.  above  then  the  values  corresponding  to  p  -  .02  and 
p  ■  .2  would  be  .687  and  .631,  respectively,  a  decline  as  before  but 
actually  a  relatively  insensitive  reaction. 

This  last  part  has  demonstrated  that  the  choice  of  combining  element 
is  greatly  influenced  by  the  probability  of  the  operating  state,  F.  In  the 
case  that  this  probability  is  very  small  (the  case  more  likely  to  actually 
occur)  the  "and"  gate  is  the  better  combining  element;  as  this  probability 
increases  the  majority  element  begins  to  be  more  promising. 
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IV.  ON  THE  USE  OP  FEEDBACK  IN  LOGICAL  NETS 
TO  PROVIDE  SELF-CORRECTING  CAPABILITIES 
by  C.  V.  Srinivasan 

A.  INTRODUCTION 

The  problem  of  developing  techniques  to  construct  reliable  logical  nets 
from  Inherently  unreliable  component  devices  is  intrinsically  of  great  value 
to  computer  designers.  Considerable  amount  of  creative  talent  has  been 
directed  in  recent  years  toward  discovering  effective  and  practical  solutions 
to  the  problem.  Surely,  there  are  many  reasons  for  interest  in  this  problem. 

It  is  reasonable  to  expect  that  in  the  not  too  distant  future  it  will 
be  technologically  possible  to  fabricate  complex  logical  nets  containing 
possibly  millions  of  logical  component  units,  each  of  which  will  be  capable 
of  operating  at  the  rate  of  many  millions  of  operations  per  second.  Such 
units  are  likely  to  be  of  extremely  small  size.  Also,  it  is  not  difficult 
for  a  computer  engineer  to  conceive  of  a  need  for  such  large  systems;  the 
biological  systems  always  provide  the  necessary  inspiration.  In  such  systems 
even  a  very  small  transient  error  probability  of,  say,  one  in  a  billion  operations 
per  component  unit,  is  likely  to  cause  millions  of  errors  in  the  calculation 
every  second.  Hence,  in  order  to  make  any  meaningful  computation  at  all,  some 
form  of  automatic  error  correction  is  necessary.  Further,  in  such  systems 
having  millions  of  components  there  will  always  exist  some  inevitable  error 
in  fabrication;  and  in  the  case  of  microminiature  components,  repair  and 
maintenance  will  require  replacements  of  basic  modules,  each  such  module 
being  a  fairly  complex  logical  net.  Therefore,  to  minimize  the  cost 
of  such  systems  it  is  necessary  to  introduce  some  redundancy  in  their  struc¬ 
ture,  so  that  a  few  failures  will  not  necessitate  a  replacement  of  an  entire 
module . 

It  is  also  true  that  the  present-day  computers  are  being  frequently 
used  to  perform  ambitious  tasks, and  oftentimes  in  such  applications  the  cost 
of  an  error  will  be  very  high.  So  it  is  very  important  to  construct  nets 
of  high  reliability.  In  satellite  systems,  for  example,  the  requirements 
on  reliability  are  particularly  severe,  as  in  such  systems  error  checking 
and  maintenance  by  a  human  being  is  impossible,  or, at  least,  is  likely  to 
be  extremely  costly.  The  above  requirements  have  only  introduced  an 
urgency  in  the  need  for  finding  a  solution  to  the  problem  of  'reliability'. 
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To  define  the  concept  of  reliability  of  a  logical  net  more  precisely, 
it  is  necessary  to  understand  the  nature  of  errors  and/or  failures  that  are 
likely  to  occur  in  such  nets  and  how  one  would  wish  to  combat  them. 

A  logical  net,  F,  having  m  input  lines  and  one  output  line,  usually 
will  perform  a  mapping  of  the  values  assumed  by  the  input  lines  to  the 
values  that  the  output  line  can  assume.  Let  us  denote  such  a  mapping  by 
the  logical  function, 

(V  =  yt  (D 


where  is  a  possible  valuation  of  the  input  lines  and  y^  is  the  corre¬ 
sponding  output  value.  The  net  might  have  been  designed  to  realize  a 
desired  logical  function,  f^,  and  because  of  the  possibility  of  erroneous 

operation  of  the  logical  components  used  in  the  net  it  is  likely  that  f*?  is 

jh  * 

sometimes  different  from  f^.  If  the  errors  in  the  net  are  transient,  (we 

shall,  hereafter,  refer  to  this  type  of  errors  as  Type  I  errors)  then  in  a 
sequence  of  successive  input-output  mappings  performed  by  the  net  (we  shall, 
hereafter,  refer  to  such  a  sequence  of  mappings  as  a  computation  done  by 
the  net)  an  input-output  pair  (X^,  y^)  that  does  not  satisfy  the  logical 
function,  f^,  is  likely  to  be  followed  by  other  pairs  that  do.  However, 
every  particular  erroneous  mapping  (we  shall,  hereafter,  refer  to  an 
individual  mapping  as  a  calculation)  performed  by  the  net  was  left  uncor¬ 
rected.  To  combat  this  type  of  error  one  would  like  to  design  nets  in 
which  the  probability,  p^,  of  occurrence  of  an  erroneous  calculation  is 
minimized.  The  smaller  this  probability  is, the  more  reliable  will  be  the 
calculations  performed  by  the  net. 

In  order  to  simplify  calculations  of  error-probabilities  in  the  case 
of  transient  errors,  it  will  be  necessary  to  make  some  assumptions  on  the 
probability  distribution  of  error-occurrence  as  a  function  of  time.  To 
begin,  we  would  like  to  assume  that  the  probability  of  occurrence  of  a 
transient  error  would  remain  a  constant  for  a  sufficiently  long  period  of 
operation  of  the  net,  as  shown  in  Fig.  9. 
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PROBABILITY  OF 
TYPE  I  ERROR 


error  in  ft  logical  net,  as  a  function  of  time. 


As  the  period  of  operation  of  the  net  increases,  due  to  an  aging 
process,  the  net  may  deteriorate,  thereby  increasing  the  probability  of 
occurrence  of  an  error.  To  combat  such  deteriorating  situations,  one  will 
have  to  develop  some  means  of  recognizing  such  deterioration  and  properly 
replace  some  components  in  the  net.  This  calls  for  entirely  different 
techniques  of  combating  errors, and  a  discussion  of  this  will  be  best  done 
under  a  different  error  category. 

Another  assumption  on  transient  error  distribution  will  be  that  the 
probability  of  occurrence  of  such  errors  in  successive  calculations  would 
be  independent.  This  assumption  is  made  primarily  for  the  purpose  of 
simplifying  calculations  to  obtain  error  estimates.  The  scheme  of  error 
correction  we  shall  employ  does  not  depend  on  this  assumption  for  successful 
operation.  However,  an  estimate  of  improvement  for  the  above  case  of  inde¬ 
pendent  probability  distribution  for  successive  calculations,  will  give  us 
some  idea  about  how  effective  the  error-correcting  had  been.  We  may,  there¬ 
fore,  write  for  Type  I  errors,  that,  for  an  input  valuation,  3^, 

(Sp  wtth  Probability  (1  -p±) 

(Si)  -  y  (2) 

l^D  (Si)  wit**  probability  p^. 

in  the  case  of  nets  where  each  input  and  output  line  assusnes  at  a  time  only 

one  of  the  two  values  0  or  1  (f^j  is  the  complement  of  f^).  Generally,  the 

probability  Pi  could  be  a  function  of  3^.  In  such  a  case  one  could  safely 

choose  the  largest  Pj^  as  the  first  measure  of  unreliability  of  the  net.  Let 

Max  p,  be  called  p'. 
i  1 
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However,  if  a  permanent  error  (we  shall,  hereafter,  call  this  Type  II 
error)  occurred  in  a  net  due  to,  say,  the  breakdown  of  a  component  in  the 
net,  then  in  a  computation  performed  by  the  net,  the  outputs  after  the 
instant  of  occurrence  of  the  error  are  likely  to  correspond  to  some  other 
arbitrary  function,  f^,  or  it  may  even  just  remain  at  a  constant  value.  In 
the  former  case,  if  one  knew  what  error  occurred  one  could  perhaps  just 
use  an  appropriate  additional  calculation  for  error  correction.  In  the 
latter  case,  of  course,  the  net  has  to  be  replaced  or  repaired  after 
the  recognition  of  the  error,  and  this  requires  that  the  iomputation  be 
temporarily  stopped.  It  is  clear  that  to  do  any  error  correction,  recog¬ 
nition  of  the  error  would  be  essential.  To  combat  this  type  of  error  it 
is  necessary  either  to  construct  nets  with  components  of  a  very 
low  probability  of  complete  breakdown  or  to  build  into  the  net  a  capa¬ 
bility  to  repair  itself.  Repair  could  be  through  error  diagnosis  and 
component  replacement  or  else  through  a  direct  substitution  of  an  alternate 
net  in  place  of  the  one  that  had  failed.  Such  error  correction  could  be 
done  also  in  anticipation  of  a  failure  due  to,  say,  an  aging  process, 
referred  to  earlier. 

If  the  probability  of  occurrence  of  a  need  for  replacement,  during  a 
calculation  is  p"  and  p"  remains  a  constant  for  a  sufficiently  long  time, 
then  one  may  choose  p"  as  the  second  measure  of  unreliability  of  the  net. 
Or,  the  expected  value  of  the  life  of  a  net  —  the  life  being  the 
period  before  the  first  repair  —  may  be  chosen  as  the  second  measure  of 
reliability. 

Two  general  techniques  of  interest  and  of  some  promise  are  at  present 

known  for  designing  logical  nets  having  a  p'  (probability  of  occurrence  of 

a  Type  I  error)  less  than  that  of  the  component  parts  used  in  the  nets. 

These  two  schemes  also  improve,  to  a  certain  extent,  the  second  measure  of 

3 

reliability  —  vie. the  life  of  the  net.  The  first  is  von  Neumann's 
major! ty-vote- taker  scheme  and  the  second  is  the  triangular  recursive 
scheme1-^ developed  at  this  laboratory.  In  both  cases  errors  in  calcu¬ 
lation  are  masked  by  the  superfluous  calculations  that  tend  to  reinforce 
preferentially  the  probability  of  generation  of  the  correct  output.  Both 
these  schemes  do  not  offer  the  capability  to  detect  and  diagnose  the 
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occurrence  of  errors.  One  might,  therefore,  say  that  these  schemes  are,  in 
a  sense,  passive.  The  second  scheme  is  even  further  restrictive,  because  it 
will  not  result  in  an  improvement  of  reliability  in  all  possible  logical 
functions,  ("not"  function  is  a  typical  example  where  the  triangular  recursive 
scheme  is  not  applicable.)  However,  for  those  functions  for  which  it  is 
applicable,  the  triangular  scheme  will  yield  greater  improvement  in  reliability 
than  the  majority-vote-taker  scheme.  Thus,  by  the  proper  use  of  both  the 
schemes,  one  may  introduce  a  considerable  improvement  in  the  reliability  of  a 
logical  net. 

Logical  nets  that  are  designed  to  serve  usefully  for  a  long  period 
of  time  should  have  some  facility  to  combat  the  effects  of  deterioration 
of  component  units  due  to  aging  or  other  external  interference.  One  would 
further  like  to  have  such  a  net  take  preventive  measures  to  avoid  a  failure 
that  may  be  anticipated.  Such  features  will  extend  the  expected  life  of  a 
net.  The  passive  schemes  described  above  will  not  provide  these  capabilities. 
Some  form  of  active  error  correction  through  error-detection  and  -repair  is 
definitely  called  for  to  extend  the  useful  life  time  of  a  logical  net. 

Such  a  self-correcting  net  would  have  to  be  basically  a  sequential 
net  with  a  finite  memory.  The  general  problem  of  automatic  error  diagnosis 
and  repair  in  logical  nets  with  memory,  seems  to  be  a  very  difficult  problem: 
at  least,  the  presently  available  concepts  appear  to  be  inadequate  to  develop 
and  analyze  significant  techniques.  In  this  report  we  present  a  simple 
scheme  which,  in  a  sense,  is  self-correcting.  However,  the  correction  is 
achieved  through  the  substitution  of  a  new  unit  in  place  of  another  that 
produced  an  erroneous  calculation  and  not  through  error  diagnosis  and  repair 
at  the  component  level.  In  the  context  of  this  model,  the  problems  that 
arise  in  the  design  of  such  nets  are  pointed  out.  This  method  of  correction 
may  be  uniformly  applied  to  all  logical  nets,  as  long  as  new  substitutes  are 
easily  available.  It  is  our  contention  that  diagnosis  and  repair  at  the 
component  level  are  essentially  the  properties  of  systems  that  possess  the 
facility  for  practically  unlimited  growth  —  we  have  in  mind  systems  that  can 
produce  practically  unlimited  quantities  of  the  basic  cells  with  which  they 
are  made. 
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Clearly,  for  a  given  component  reliability,  every  improvement  achieved 
in  the  reliability  of  a  net,  composed  of  the  given  components,  has  to  be  paid 
for  in  some  form  or  other.  In  the  passive  schemes  the  designer  is  required 
to  pay  in  terms  of  the  increased  number  of  components  used  in  each  net,  and 
the  consequent  increased  delay  in  operation.  However,  the  designer  does  not 
have  much  control  over  the  amount  of  time  that  a  net  will  consume  to  perform 
an  operation.  Further,  the  amount  of  time  delay  will  remain  the  same  irre¬ 
spective  of  whether  an  error  occurred  during  a  calculation  or  not.  In  the 
case  of  error  detection  and  repair  also  the  designer  will  have  to  pay  in 
terms  of  greater  number  of  components  and  greater  time  for  computation.  But, 
in  this  case,  the  time  delay  may  be  used  more  efficiently.  For  instance,  the 
length  of  a  computation  will  be  large  only  in  case  an  error  occurred;  other¬ 
wise  the  delay  will  be  small.  Also,  the  delay  time  may  be  used  for  error- 
diagnosis  and  error- signalling.  As  the  component  units  deteriorate  in  a 
large  logical  system  of  self-correcting  subnets,  reliability  will  not  decrease. 
However,  more  time  will  be  needed  for  computation. 


B.  THE  DESCRIPTION  OF  THE  MODEL 

To  begin  with,  we  shall  confine  our  attention  to  nets  having  a  single 
output  line  and  arbitrary  nunber  of  input  lines.  Each  line  will  be  capable 
of  assuning  only  one  of  the  two  possible  values  at  a  time,  namely  0  or  1 . 
Each  net  will  consist  of  two  parts,  as  shown  in  Figure  10. 


FORWARD  PATH 

Xj  - 


Fig.  10  The  Block  Diagram  of  a  Self-Correctina 
Net. 
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The  part  F  is  in  the  forward  path  and  the  part  G  is  in  the  feedback  path. 

The  delay  units  represent  the  intrinsic  delays  involved  in  the  calculations 
of  F  and  G,  and  in  addition,  may  also  have  additional  periods  of  delay  intro¬ 
duced  for  the  purpose  of  synchronization  with  a  master  controller,  or  for 
whatever  other  reasons  that  might  exist.  The  feedback  net  G  is  the  error- 
correcting  net.  For  each  input  valuation  of  the  net  F,  the  net  G  will 
decide  whether  the  input-output  pair,  (X^,  y^),  satisfies  a  desired  logical 
function  fp  and  generate  an  appropriate  control  signal,  x^,  that  would  con¬ 
dition  F  suitably  for  the  next  calculation.  Of  course,  the  net  G  is  also 
likely  to  generate  erroneous  outputs.  However,  when  z^  is  correct  its  value 
will  indicate  whether  the  pair  (X^,  y is  a  proper  pair  or  not. 

For  a  given  fixed  input,  X^,  the  net  in  Figure  10  will  produce  a  compu¬ 
tation  (a  sequence  of  calculations)  consisting  of  pairs  of  outputs  (y^,  z^). 
In  this  computed  sequence,  the  value  of  z^  in  each  pair  will  indicate  whether 
the  corresponding  y^  is  the  correct  output  or  not.  In  general,  instead  of 
keeping  the  input,  X^,  fixed  it  could  be  encoded  into  an  input  sequence  and 
in  this  case  a  sub-sequence  of  the  computer  output  sequence  would  be  the 
required,  correct  output  sequence.  In  the  following  discussion  we  shall 
assume  that  the  input  is  in  the  uncoded  form  —  i.e*  is  held  fixed  for  a 

given  computation.  It  is  our  opinion  that  computation  with  encoded  inputs 
would  not  cause  any  additional  advantage  in  reliability  of  operation. 

Let  the  mappings  performed  by  the  F  and  G  nets  be  denoted  by  the 

m  |  J  p  |  | 

logical  functions  f^,  and  f^  ,  where  m  of  the  inputs  are  external  inputs, 


the  (nri-1  )st  being  the  feedback  input: 

ff**'  0^*  zt)  -  yt  (3) 

yt)  -  zi  (4) 

The  functions  F  and  G  may  be  expanded  as  follows*: 

fp  (3^,  Zj)  ■  8^(2^)  *  z£  +  8j2^)  ’  (5) 

fGWW  <*i>  Xi>  -  8“l(*i>  •  *i  +  822(*i>  *  (6) 


f •*  denotes  logical  AND,  logical  OR  and  denotes  complementation. 
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where  the  g's  are  functions  of  m  arguments.  To  specify  completely 
the  computation  performed  by  such  a  finite  sequential  machine  it  is  necessary 
to  fix  a  distinguished  initial  state  (namely,  the  contents  of  the  delay  units 
at  time  0)  and  an  end  condition  for  stopping  the  calculations.  Also,  the 
sequence  of  output  pairs,  (y,z),  should  be  properly  indicated  as  functions  of 
time. 


Assuming  that  the  delay  introduced  by  the  delay  units  in  Figure  10  is 
of  unit  magnitude  and  the  net  as  a  whole  is  periodically  activated  to  perform 
a  calculation  at  the  beginning  of  every  unit  interval  of  time,  the  equations 
(5)  and  (6)  may  be  written  as  follows: 


•  Vi  +  •  V. 


and 


zn  *  b1n<V  *  Vi  +  b2n<*i>  *  V, 


(7) 

(8) 


where  n  *  1,  ...  is  the  time  parameter,  z^,  y^  are  the  n^  values  of  the 
outputs,  and  a1n,  a2n,  bJn  and  b2n  represent  the  'g'  functions  of  equations 
(5)  and  (6)  at  the  instant  n.  In  general,  it  is  thus  possible  that  as  n 
varies  the  g  functions  may  also  vary.  Let  us  now  see  how  the  initial  and 
final  conditions  may  be  specified  for  the  equations  (7)  and  (8)  and  how  the 
resultant  computation  may  be  used  to  calculate  a  desired  logical  function 


The  end  conditions  may  be  specified  in  one  of  two  possible  ways.  One 
can  fix  the  length  of  computation  for  a  given  input  to  be  the  same  number, 

N,  and  specify  a  rule  for  selecting  the  appropriate  output  from  the  N 
calculations.  This,  of  course,  will  require  additional  equipment  to  perform 
the  selection;  and,  further,  in  this  scheme  one  does  not  take  advantage  of  the 
probabilities  of  distribution  of  occurrence  of  errors  to  increase  the  speed 
of  calculation.  In  fact,  the  selection  procedure  will  be  consuming  additional 
time  for  completing  the  calculation. 


On  the  other  hand,  one  may  decide  to  terminate  the  calculations  for 
a  given  input,  on  some  criterion  depending  upon  the  relative  values  of  the 
outputs  (y,s).  There  are  only  three  distinct  possibilities  in  which  this 
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may  be  done,  and  these  are'*:  (Note  that  the  corresponding  outputs,  as  shown 
in  Figure  9,  are  always  separated  by  one  unit  of  delay.) 


(1) 

yn 

2  n+1 

yn  “  fD«> 

(2) 

zn+1 

-  1  -yn 

■ 

(3) 

2  n+1 

-  °-yn 

Permutation  of  y  and  z  in  the  above  three  conditions  does  not  change  the 
essential  nature  of  the  constraints. 

In  case  (1)  an  external  comparator  is  necessary  to  determine  that  and 

z  ,  are  of  the  same  value  and  generate  the  appropriate  end  signal. 
n+1 

The  remaining  cases  presume  the  existence  of  some  form  of  verification  within 
the  net;  and,  hence,  the  proper  value  of  zn+]  may  itself  be  used  as  the 
terminating  signal.  In  all  the  three  cases  an  erroneous  output  yn  j*  fp(X) 
will  be  chosen  only  if  both  y  and  z  are  simultaneously  in  error.  Also,  for 
a  given  input,  calculations  will  be  continued  only  as  long  as  there  is  an 
error  in  the  output;  and  this  feature,  if  properly  used,  will  result  in  the 
saving  of  time. 

In  the  following  discussions  we  shall  consider  only  cases  (2)  and  (3) 
for  the  end  condition,  as  these  seem  to  take  into  account  more  comprehensively 
than  the  rest  all  the  equipment  needed  for  the  calculation.  It  should,  of 
course,  be  noted  that  in  our  discussions  so  far,  we  have  implicitly  assumed 
that  there  is  an  external  master  controller  that  provides,  periodically,  the 
clocking  pulses  needed  for  starting  each  calculation. 

Also,  as  the  length  of  every  computation  is  not  the  same, it  is  necessary 
to  introduce  a  facility  to  gate  out  the  proper  result  at  the  termination  of 
a  computation  and  to  gate  in  a  new  set  of  inputs  for  the  next  computation. 

In  cases  (2)  and  (3)  of  the  end  condition,  the  terminating  value,  z^, 
may  be  used  to  start  the  gating  operations  at  the  input  and  output  ends  of 
the  unit.  In  such  an  arrangement  the  terminating  value,  zR,  may  also  be  the 
initial  value,  zq,  for  the  next  computation.  Let  the  initial  value  of  y, 
namely  yQ,  be  chosen  to  be  the  same  as  the  value  of  the  previous  output. 
Therefore,  yQ  may  be  either  1  or  0.  This  arrangement  eliminates  the  necessity 

*  The  reader  may  easily  verify  this  assertion. 
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for  resetting  the  initial  values  at  the  beginning  of  each  computation.  The 

problem  now  is  to  get  the  conditions  on  the  functions  a^,  a2n>  and  b2n  in 

order  to  calculate  a  desired  function,  fp,  with  the  above  model  of  a  sequential 

net,  using  the  smallest  necessary  number  of  calculations.  We  shall  see  that 

the  imposition  of  the  restriction  of  minimum  computational  delay  forces  the 

functions  a,  ,  a_  ,  b.  ,  and  b_  to  assume  the  value  of  either  fT(X)  or 
l  n  in  I  n  *n  u  w 

fJ(X),  for  all  n. 


c.  DETERMINATION  OF  THE  FUNCTIONS  a  ,  a  ,  b  ,  b 

In  2n  In  2n 

Before  proceeding  with  the  analysis  let  us  take  stock  of  the  assumptions 
we  have  made  so  far,  and  the  limitations  these  assumptions  will  impose  on  the 
general  validity  of  whatever  conclusions  we  may  arrive  at. 

We  have  now  three  requirements  on  the  computation  performed  by  the  net 
of  Figure  10.  These  are 

(i)  The  net  should  start  each  computation  from  one  of  the  prescribed 
initial  states.  (Notice,  however,  that  each  calculation  of  the  net  might 
start  from  any  one  of  the  four  possible  initial  states). 

(ii)  A  computation  is  terminated  when  the  end  condition  is  satisfied. 

At  the  termination  of  a  computation  the  selected  output  value  should  be  equal 
to  fD(X) . 

(iii)  In  order  to  minimize  computational  delay  the  net  should  seek  to 
satisfy  the  end  conditions  in  each  calculation.  Also,  after  termination,  the 
selected  output  of  the  computation  must  be  equal  to  f^(X). 

The  initial  state  of  a  computation  was  chosen  to  be  the  same  as  the 
terminating  state  of  the  previous  computation.  This  was  possible  because 
the  terminating  state  was  used  only  to  do  the  proper  gating  at  the  input  and 
output  ends,  and  not  to  terminate  the  operation  of  the  net  itself.  The  other 
possibility,  namely  resetting  the  initial  state  to  a  fixed  distinguished 
value  at  the  beginning  of  each  computation  will  require  additional  equipment 
to  do  the  resetting  operations.  This  will  only  increase  the  complexity  of  the 
net,  thereby  reducing  its  reliability  of  operation.  Therefore,  the  choice 
we  made  is  preferable. 
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Out  of  the  three  possible  end  conditions,  the  latter  two  take  into 
account,  more  comprehensively  than  the  first,  all  the  operations  that  the 
net  should  perform  to  complete  a  computation.  Therefore  we  shall  confine 
our  attention  only  to  the  end  conditions  (2)  and  (3).  In  fact,  if  an 
external  comparing  device  is  introduced  in  the  net  G  of  Figure  ID,  for  the 
end  condition  (I),  to  generate  the  terminating  signal,  then  the  end  condition 
(1)  will  reduce  to  one  of  the  remaining  two  conditions.  Thus,  analysis  of 
the  net  for  conditions  (2)  and  (3)  should  exhaust  all  the  possible  termi¬ 
nations  with  which  the  net  might  be  operated. 

The  third  requirement  is  necessary  to  satify  the  condition  that  each 
computation  should  be  of  the  shortest  possible  length.  If  this  requirement 
is  not  imposed,  it  is  not  clear  a  priori,  what  function  the  net  should 
calculate  in  each  operation."' 

The  operation  of  the  net  satisfying  the  above  three  requirements,  for 
the  end  conditions  (2)  and  (3)  is  described  by  tables  I  and  II,  respectively. 
These  tables  may  be  read  as  follows: 


Table  I: 

(i) 

Design  Requirement: 

Each  computation  should  be 
possible  length. 

of  the  shortest 

(ii) 

End  Condition: 

Vh"1- 

yn  *  fD(*i>* 

(iii) 

Initial  Condition: 

yQ  -  0  or 

1  and  z  »  1 . 
o 

yn 

z 

n 

w 

yn+1 

zn+1 

*!„<*!> 

b.n<V 

b2n% 

0 

1 

1 

1 

0 

1  * 

<* 

0 

0 

1 

0 

0 

1 

0  * 

* 

1 

1 

1 

1 

1 

1 

1  * 

1 

* 

1 

1 

0 

0 

0 

o  * 

0 

* 

0 

0 

1 

1 

0 

*  1 

* 

0 

0 

0 

0 

0 

1 

*  0 

* 

1 

1 

0 

1 

1 

1 

1 

1 

ii 

1 

0 

0 

0 

0 

*  0 

0 

'Jx 

*  It  is  likely  that  the  function  to  be  calculated  by  a  net,  in  each 
calculation,  may  be  determined  through  some  conditions,  connected  with  error 

diagnosis.  In  such  a  case  the  requirement  that  each  computation  be  of  the 
minimum  possible  length,  will  not  be  satisfied. 
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Table  II: 


(i) 

Design  Requirement: 

Same 

as  Table  I 

(ii) 

End  Condition: 

2n+1 

-  0  -  yn  " 

(iii) 

Initial  Condition: 

yo  " 

0  or  1  and  z  *  0. 

0 

yn 

z 

n 

w 

yn+1 

Zn+1 

a,n<*i>  a2n<*i>  b 

b2n<*i> 

0 

0 

1 

1 

1 

*  1 

* 

1 

0 

0 

0 

0 

0 

*  o 

* 

0 

1 

0 

1 

1 

0 

*  1 

0 

* 

1 

0 

0 

0 

1 

*  0 

1 

* 

0 

1 

1 

1 

1 

1 

* 

1 

0 

1 

0 

0 

0 

0 

0 

1 

1 

1 

1 

0 

1 

0 

* 

1 

1 

0 

0 

1 

0 

1 

☆ 

Let  us  first  consider  the  Table  I.  The  first  two  colianns  of  this 
table  give  the  values  of  the  outputs  at  the  instant  n,  for  n  -  0,  1 ,  ... 

The  third  column  gives  the  value  of  f^(X^),  for  the  input  at  the  instant 
n.  The  fourth  and  fifth  columns  give  the  expected  values  of  the  outputs 
at  the  next  instant,  to  satisfy  the  requirements  that  the  proper  computation 


should  be  of  the  shortest  possible  length.  For  these  input  and  output  con¬ 
ditions  one  may  calculate  from  equations  (7)  and (8)  the  values  that  the  functions 
'a'  and  'b1  must  assime,  for  the  input  X^* 


Consider,  for  example,  the  first  rov  of  Table  I.  For  the  initial  con 
ditions  shown,  one  gets  from  equations  (7)  and  (8) that  for  n  ■  T, 


T ' 1  "  a,T(£>  "  1 

(9) 

T-H  “  b2T(-)  "  0 

(10) 

and  Sg^X)  and  b^(X)  might  assume  any  values  whatsoever.  As  ■  0  the 

end  condition  is  r.rt  satisfied,  and,  therefore,  the  net  goes  through  another 
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calculation  for  the  same  input  X. ;  only  it  starts  from  the  initial  state, 

"1 

(y,z)  *  (1,0),  instead  of  (0,1).  The  operation  of  the  net,  this  time,  is 
described  by  the  last  but  one  row  of  Table  I.  The  final  outputs  are,  f from 
equations  (7)  and  (3)] 


yT+2  *  a2(T+1)^i*  1 

ZT+2  “  b1(T+1)(^i^  " 

and  these  do  not  depend  on  the  values  assumed  by  aj  )  ^7  and  b2(T+1)^P’ 

as  indicated  by  the  in  the  table.  The  output  pair  (yT+j>  zx+2^  nOW 

satisfies  the  end  condition  for  Table  I  and  as  such  the  computation  is  term!* 
nated.For  other  initial  states  the  table  is  similarly  filled  in.  Table  II 
is  constructed  similarly,  with  the  only  difference  that  the  end  condition 
is  now  different.  The  following  observation  concerning  the  operations  shown 
in  Tables  I  and  II  are  now  pertinent. 

1.  Comments  on  Tables  I  and  II 

Comment  1  The  value,  z^-1 ,  in  Tables  I  and  II  fixes  the  value  of 
ajn(X^)  =  f°r  all  X^.  When  z n  •  0,  a^CX^)  would  still  have  the 

same  value,  unless,  of  course,  the  net  corresponding  to  a^n  was  itself 
changed  by  the  change  in  the  value  of  z^.  However,  when  z n  ■  0,  the  output 
is  independent  of  a^QC^),  as  ®ay  ke  verified  from  equation  (7).  So  one 
would  find  no  reason  to  modify  the  net  of  a^  as  a  function  of  z .  Thus, 
the  net  may  be  kept  fixed  for  all  time,  n;  no  additional  advantage  will  be 
gained  by  making  the  net  adjustable. 

Similarly  z  ■  0  fixes  the  function  a,  s  fn  for  all  X.  and  for 
n  Zn  D  -t. 

all  time  n,  and  in  this  case  (z  ■  0)  the  output  is  independent  of  a,  (X,). 

ft  i  ft 

Therefore,  if  two  separate,  identical  nets  were  used,  one  for  a  and 

1  ^ 

another  for  ajj,,  then  depending  upon  the  value  of  zn  the  output,  y^,  will 
depend  on  one  or  the  other  of  the  two  nets.  The  input  *n  may  be  used  to 
switch  the  operation  from  one  net  to  the  other  depending  upon  which  one 
produced  a  wrong  output.  This  feature  introduces  the  self -correcting 
property  in  nets  of  this  type. 
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Likewise,  yn  *  0,  fixes  b2n  s  fjj  in  Table  I  and  b2n  *  fp  in  Table  II; 

y  ■  1,  sets  b,  =  f_  in  Table  I  and  b,  =  F,,  in  Table  TI, 

'n  ’In  D  In  D 

For  these  values  of  ’a'  and  'b*  functions  equations  7  and  8  may  be 
written,  for  the  end  condition  (2)  as 

V-1  ’  •  *a  +  fD®  '  <l3> 

*„+,  ’  S®  '  ».  +  ED®  •  <'*> 

and  for  the  end  condition  (3), 

Vfl  *  £„®  '  ■„  +  S®  •  05) 

an<l  znrl  ■  £D®  '  yn  +  £D®  '  ya  <>6> 

The  use  of  the  same  function  ffl  more  than  once  in  the  above  equations  signifies 

the  fact  that  each  value  would  be  calculated  by  a  distinct  logical  net. 


The  above  values  of  the  functions,  to  be  calculated  by  each  net  in 
each  calculation,  were  obtained  because  of  the  restrictions  imposed  on  the 
required  values  of  y  j  and  z  ^  in  columns  4  and  5  of  Tables  I  and  II.  The 
values  of  yn+j  and  were  chosen  for  each  calculation  such  that  the  whole 

computation  will  be  terminated  with  the  shortest  possible  delay,  producing 
the  correct  output  *  f^J(X).  One  may  easily  verify  that  the  above  assign¬ 
ment  is  the  only  possible  one,  that  would  minimize  the  length  of  each 
computation. 


Comment  2.  For  no  two  distinct  initial  states,  (v  .  z  >  the  same 
-  '■'n'  n' 

pair  of  functions  (a^,  kjn)»  (i,j  -  1  or  2)  is  used  in  the  calculations. 
Therefore,  unless  an  erroneous  operation  produces  at  the  instant  (n  +  1 ) 
the  same  erroneous  outputs  as  at  the  instant  n,  a  new  calculation  is 
always  done  using  a  new  net. 

Comnent  3.  The  equations  (14)  and  (16)  represent  just  modulo  2  com¬ 
parison  functions,  which  may  be  written  as 
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and 


zn+1  “  fD<*>  @  yn 

zn+1  “  fD^  ^  yn’ 

where  ($  indicates  the  'Exclusive  OR'  operation.  The  feedback  network 
therefore,  compares  the  value  of  yn  with  the  required  value,  fp(X),  and  gives 
a  command  for  recalculation  whenever  yn  •  fp(X). 

In  fact,  in  the  discussion  in  section  B-,  on  the  model  shown  in  Figure  6 
we  had  required  of  the  net  some  such  similar  operation.  There  we  said  that 
the  feedback  input  would  condition  the  net  suitably  for  the  next  calculation. 
The  above  arguments  indicate  that  the  only  possible  useful  conditioning  that 
may  be  done  will  be  to  give  a  command  for  recalculation,  possibly  using  an 
alternate  net.  This  conclusion  will  be  valid,  of  course,  only  if  the  require¬ 
ment  of  minimum  computational  delay  is  imposed  on  the  net.  At  present,  it  is 
not  clear,  what  other  general  requirement  may  be  substituted  in  its  place. 

The  initial  and  final  conditions  were  specified,  in  a  sense,  in  a  very  natural 
way.  In  this  case,  it  is  clear  that  there  are  no  other  boundary  conditions 
that  could  be  more  profitably  used. 

We  believe  that  even  in  the  case  of  computation  with  encoded  inputs 
the  same  conclusion  would  be  valid.  However,  it  is  difficult  to  prove  this 
conjecture  in  this  generality,  because  it  is  not  clear  what  general  restric¬ 
tions  exist  on  codes  for  computation.  (In  an  analogous  situation  in  the  case 
of  communication  systems,  a  well-known  set  of  general  restrictions  on  possible 
efficient  codes  for  error-correction  is  provided  by  the  properties  of  group 
codes) . 

2.  Generalization  of  the  Concepts: 

The  above  arguments  can  be  directly  extended  to  the  case  where  there  is 
more  than  one  output  line  for  each  net  F  and  G  of  Figure  10.  Let  the  output 
lines  of  F  be  denoted  by 

Z  -  (y,  y2  •••  yt)  >  c  >  1 


<14)a 

(16)a 
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At  any  instant  n,  if  the  output  valuations  were  (n)  and  z^(n). 
0  <  i,  j  <  2C-1  then  from  equations  (17)  and  (18)  we  get: 


(n+1) 


*kj  n  )  k  “  *  >  •  •  •  t , 


zk  (n+1 ) 


^kjn  m  ''  *  * '  c  » 


where  X  would  be  the  input  at  the  instant  n.  Thus  for  each  pair  of  output 

vectors  (y^,  Zj)  the  two  nets  corresponding  to  the  functions  a^n  and  b^in 

are  chosen  for  calculation,  for  each  k  •  1,  .  ..t.  For  the  same  reasons 

discussed  in  section  C.1,  comment  1,  the  functions  'a'  and 'b'  may  be  chosen 

to  be  independent  of  the  time  parameter,  n.  Also,  for  each  k,  1  <  k  <  t 

if  the  desired  output  function  is  f «  (X).  Then  from  requirement  3"we  will 

*Tc 

obtain  that 


a  m  f 
kj  D. 


for  all  j  «  0,  1,  ...,  2C-1,  and  all  n  *  1,  2,  ... 


For  any  given  value,  y^(n),the  value  ^ (n+1 ),  may  be  used  to  indicate 
whether  ^(n)  is  correct,  and  if  it  is  not  correct,  which  one  of  the  2C-1 
possible  errors  is  present  in  y^(n).  In  this  case,  therefore,  the  design 
of  the  nets  corresponding  to  the  functions  b^  will  depend  on  the  nature 
of  error  diagnosis  that  is  desired.  It  should  be  valuable  if  some  general 
conditions  could  be  obtained  on  the  specification  of  the  functions 

It  is  not  clear  at  present,  what  the  error -correcting  potential  of 
this  scheme  will  be,  and  taking  into  consideration  all  the  extraneous 
control  nets,  what  the  overall  gain  in  reliability  will  be. 

The  values  of  the  probability  of  occurrence  of  a  proper  computation 
and  that  of  an  erroneous  computation  for  some  typical  nets  are  calculated 
in  the  next  section.  Also,  the  expected  length  of  a  proper  computation, 
as  well  as  an  improper  one,  are  obtained.  We  shall  discuss  the  results  and 
their  significance,  in  detail,  in  the  next  section.  The  following  general 
consents  may  be  made  concerning  the  scheme,  at  this  moment: 

Minimus  computational  delay 
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It  is  clear  that  the  process  of  substitution  of  a  new. net  in 
place  of  another  that  generated  an  erroneous  output,  indicates  that,  with 
this  scheme  it  should  be  possible  to  combat  both  type  1  (temporary)  and 
type  2  (permanent)  errors. 

Also,  there  is  the  possibility  of  developing  error-diagnostic 
nets  which  will  prove  to  be  of  great  value  in  any  large  data  processing 
system. 

It  appears  that  if  more  redundant  alternate  nets  were  available 
in  a  net,  then  greater  reliability  of  operation  would  be  realizable.  Also, 
the  principal  concept  of  substituting  an  alternate  net  in  place  of  an  erroneous 
one,  is  very  elementary  in  error-correction.  The  above  scheme  seems  to  offer 
a  natural  way  of  doing  this,  in  practice. 

We  believe  that,  the  model  discussed  so  far,  provides  a  good  context 
in  which  further  investigation  may  be  made  to  understand  the  process  of 
self-correction  and  possibly  develop  general  theoretical  concepts  on  error- 
diagnosis  and  self-repair.  We  shall  be  interested  in  answering  questions 
of  the  following  nature: 

1 .  For  a  given  number  of  feedback  lines  what  boundary  conditions 
will  yield  the  maximum  reliability?  The  selection  of  an  appropriate 
boundary  condition  would  be  influenced  by  the  following  considerations: 

The  number  of  additional  operations  necessary  to  termi¬ 
nate  one  computation  and  start  the  next  computation. 

The  nature  of  error-signalling,  if  any,  that  is  expected 

of  the  net. 

The  cost  of  termination  in  terms  of  loss  in  reliability. 

2.  If  the  precise  nature  of  possible  errors  were  known, is  there 
any  significance  in  changing  the  logical  function  for  different  calculations 
in  a  computation? 

In  the  above  discussion  we  assumed  the  logical  function  for  each 
calculation  to  be  fD(X) .  By  changing  the  logical  function  for  a  given 
net,  it  may  be  possible  to  perform  error-diagnosis. 
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3.  In  a  large  net  composed  of  subnets  of  the  form  discussed  so 
far,  the  total  net  is  essentially  asynchronous ,  because,  depending  upon 
the  occurrence  of  errors,  each  subnet  may  take  a  different  time  to  complete 
its  computation.  This  calls  for  a  proper  scheduling  of  each  operation  or 
provision  for  buffer  storages  at  the  input  of  each  subnet.  Depending  upon 
the  probabilities  of  occurrence  of  errors, a  queuing  problem  arises.  The 
length  of  the  queue  at  the  input  of  each  subnet  will  directly  be  a  measure 
of  the  performance  of  the  net.  It  remains  to  be  seen  how  this  property 
can  be  exploited  to  do  error-prediction  in  a  systematic  way.  In  the  design 
of  a  large  net  oneb  aim  should  be  to  minimize  the  length  of  the  queue  at 
the  input  of  each  subnet. 

4.  In  every  net  of  this  form  a  possibility  exists  that  the  control 
net  might  fail,  causing  a  computation  to  be  continued  indefinitely  without 
termination.  To  combat  this  type  of  failure  one  may  assign  a  maximum  length 
N  for  each  computation.  After  N  calculations,  it  will,  therefore,  be  neces¬ 
sary  to  adopt  some  type  of  statistical  estimate  to  choose  the  correct 
output.  A  study  of  this  will  also  be  a  part  of  a  study  of  self-correcting 
system. 


These  problems  arise  in  a  natural  way  in  the  context  of  the  self- 
correcting  scheme  discussed  in  this  report.  It  is  our  opinion  that  a  study 
of  these  problems  in  greater  depth  would  be  a  worthwhile  task. 


D.  THE  ANALYSIL  OF  DYNAMIC  BEHAVIOR  OF  A  SELF-CORRECTING  NET 

The  dynamic  behavior  of  a  self-correcting  net  may  be 
described  by  the  probabilities  of  occurrence  of  the  following: 

(1)  A  proper  computation  of  length  /. 

(2)  An  erroneous  computation  of  length  l. 

(3)  A  computation  being  of  length  greater  than  l. 

Let  the  respective  probabilities  of  events  (1),  (2),  and  (3)  be  denoted  by 
Pp(i),  <P  stands  for  ’proper1),  P e</),  (e  for  erroneous)  and  Pc(i),  (c  for 
continuation).  The  overall  performance  of  the  net  may  be  evaluated  from: 
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(21) 


(22) 


where  P  is  the  total  probability  of  occurrence  of  a  proper  computation  of 
any  length,  and  so  also,  Pe,  that  of  an  erroneous  computation.  It  is  clear 

that. 


Pc(i  -  0  -  BpU)  +  PeU>  +  Pc(*>, 


(23) 


and,  therefore,  from  equations  (21),  (22)  and  (23). 

00  CO 

£  Pc(i  -  1)  Pc(i>  -  Pc(0)  -  Pe  +  Pp  -  1  • 
i-1  i-1 


(24) 


Also,  it  is  true  that 

,<£•„  »CW  '  0  (25) 

The  expected  values  and  variances  of  the  lengths  of  a  proper,  as  well  as  an 

erroneous  computation  are  of  significance  in  evaluating  the  performance  of 

such  nets.  Let  E(ip)  and  E(ie>  denote  the  respective  expected  values.  So 

also,  let  V(f  )  and  V(i  )  denote  their  variances: 

P  e 


09 

(26) 

E(/p)  -  { ^ 

/•I 

00 

/  •  pp(i))  /  pp 

E(/e)  *  ( ^ 

00 

i  •  Pe(Z)}  /  Pe 

A 

(27) 

v<v  ■  (I 

/-I 

00 

[i  -  E(/p>r  pp(i»  /  pp 

(28) 

«v  ■  r[ 

/•i 

ti  -  E(/e)]2  Pe(x;>)  /  Pe. 

(29) 
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For  given  component  reliabilities  a  good  design  for  a  self-correcting  net 

should  minimize  P  .  E(i  )  and  V(i  ).  Also  it  will  be  desirable  to  have 
e  p  p 

E(Je)  >  E(ip).  (30) 

In  fact,  the  most  desirable  situation  will  be: 

E(2e)  -7v<V  >  E(y  (31) 

The  above  conditions  will  assure  us  that  a  proper  termination  of  a  compu¬ 
tation  is  always  more  likely  to  occur  than  an  erroneous  one. 

ffe  shall  now  discuss  the  analytical  techniques  that  might  be  employed 
to  calculate  the  above  statistical  parameters  of  a  self-correcting  sequential 
net. 

1.  Temporary  Errors:  (Type  I) 

(a)  The  Mathematical  Model: 

In  the  case  of  temporary  errors  the  operation  of  a  self-correcting  net 
may  be  described  by  a  Markov  chain  diagram,  as  shown  in  Figure  11. 


Fig.  11  A  Markov  chain  diagram  of  a  self- 
correcting  net. 
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Each  node  in  this  figure  represents  a  possible  output  state  of  the  net  at 
an  instant,  n.  A  directed  branch,  between  a  pair  of  nodes,  indicates  a 
possible  transition,  in  the  direction  of  the  branch,  from  one  output  state 
at  an  instant  n,  to  the  other  at  the  next  instant,  n  +  J*.  A  self-loop 
indicates  that  it  is  possible  to  have  the  same  output  state  at  successive 
instants  of  time.  The  weight  on  each  branch  indicates  the  probability  of 
occurrence  of  the  transition  represented  by  the  branch.  The  probabilities 
for  5  «*  a,  £,  7,  8  and  i  *  1,  2,  3,  4,  in  Figure  U,  will  be  functions  of 
the  input  of  the  net  at  an  instant  n,  and 

2  *  1  for  §  ■  Ct,  P,  y,  and  6  (32) 

i-1 

Such  a  net  will  have  a  subset  of  states,  say  Sq,  called  the  set  of 
distinguished  initial  states.  For  any  input,  X,  a  computation  by  the  net 
will  always  start  from  one  of  the  distinguished  initial  states,  soi  e  Sq. 
Depending  upon  the  terminating  conditions,  one  or  more  of  the  output  states 
of  the  diagram  will  be  chosen  as  the  terminating  states  of  the  net.  A 
computation  by  the  net  will  always  terminate  in  one  of  the  possible  termi¬ 
nating  sates.  Let  the  set  of  all  possible  terminating  states  be  denoted  by 
Sj.  A  proper  computation  may  be  defined  in  one  of  the  following  ways: 

Definition  1 .  (Proper  computation): 

(1)  A  computation,  whose  terminating  state  s  ^  e  Sp(^) C  Sj,  where 
Sp(i)  is  the  subset  of  proper  terminating  states,  for  a  desired  output 

(2)  A  computation  having  one  or  more  occurrences  of  certain  specially 
chosen  output  states,  at  certain  well-defined  places  in  the  computed  output 
sequence,  prior  to  termination.  Let  these  distinguished  intermediate  states 
be  denoted  by  Sj(^),  for  a  desired  output 

(3)  A  computation  satisfying  both  conditions  (1)  and  (2). 


*  It  is  asstmed  that  it  takes  one  unit  of  time  to  complete  a  transition. 


"T 

i 

J 


] 


1 
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Similarly,  a  subset  of  terminating  states  corresponding  to  an  Erroneous 
computation  may  be  denoted  by  Sg(£)  C  S^.  Also,  a  computation  may  be  defined 
to  be  erroneous  if  no  state  s^  e  S^(^)  occurs  as  an  intermediate  state,  at 
the  proper  places,  in  the  course  of  a  computation,  the  sets  Sp(^),  Sg(y), 
Sj(x)  and  will  satisfy  the  following  set  relationships: 


SP 

(x)  U  SE 

(z)  -  sT 

(33) 

SP 

(X)  nSE 

(z)  *  ♦* 

(34) 

and 

SI 

(z)  n  sT 

-  *  . 

(35) 

The  set,  Sq,  of  distinguished  initial  states  may  be  chosen  arbitrarily. 

For  a  given  input,  X if  ^(S^)  ■  y.t(fD  *8  t*>e  desired  function), 
then  the  probabilities  of  individual  transitions  in  the  Markov  chain  diagram 
of  the  net  will  change  as  a  function  of  such  that  the  probability  of 
occurrence  of  a  proper  computation  would  be  maximized.  As  the  number  of 
states  in  the  diagram  increases  the  nunber  of  possible  paths  for  proper,  as 
well  as  an  erroneous  computation  will  increase  rapidly.  The  variety  of  ways 
in  which  the  different  subsets  of  equations  (34)  and  05) may  be  specified 

will  be  very  large  too.  In  such  cases,  it  will  be  desirable  to  find  some 
general  conditions  on  the  selection  of  the  subsets  that  will  maximize,  F 
and  satisfy  the  inequality  (31).  At  present,  we  do  not  have  a  good  under¬ 
standing  of  this  problem.  However,  it  is  clear  that  a  theory  of  self- 
correcting  systems  must  be  capable  of  answering  such  questions. 

(b)  Techniques  for  Analyzing  Model  (A) 

Convenient  techniques  for  obtaining  the  probability  of  occurrence  of 

a  specified  event  in  a  stationary  Markov  chain  have  been  developed  by 
18 

K.  Kaplan  and  J.  Sklansky  of  RCA  Laboratories.  In  this  note  we  shall 
briefly  explain  their  procedures  in  order  that  the  reader  may  understand 
easily  the  calculations  that  are  to  follow  : 

A  Markov  chain  diagram  may  be  Interpreted  as  a  signal  flow  graph. 

where  each  branch  weight  will  denote  the  signal  gain  provided  to  the 

signal  that  is  transmitted  through  the  path  represented  by  the  branch. 

*  In  case  the  condition  1  of  Definition  1  is  not  used  this 
property  need  not  be  satisfied. 
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Ac  each  node,  Che  signals  coining  inco  Che  node  Chrough  che  various  pachs 
of  Che  graph  are  added  up  and  reCransmiCCed  chrough  Che  ouCgoing  paChs  of 
Che  node.  The  sum  of  Che  incoming  signals  is  always  equal  Co  Che  sun  of 
Che  ouCgoing  signals. 

In  such  a  graph,  if  a  uniC  impulse  is  injecCed  inCo  a  node,  say  Nq, 
from  an  excernal  source,  aC  cime  0,  one  may  calculaCe,  using  Che  z-Cransform 
Cechniques,  Che  magniCude  of  Che  signal  ChaC  will  be  presenc  aC  any  given 
node,  say  N^,  aC  a  given  insCanC,  say  n,  because  of  signal  Cransmissions 
Chrough  Che  various  pachs  of  Che  graph.  This  magniCude  of  Che  signal  ac 
N^,  ac  Che  insCanC  n,  will  be  equal  Co  che  probabilicy  of  occurrence  of  Che 
following  evenc,  in  Che  corresponding  Markov  chain; 

The  evenC  ChaC,  in  an  experimenC  on  Che  Markov  chain,  sCarCing  ac  Cime 
0,  from  a  sCaCe,  say  sq,  represented  by  Che  node  Nq  in  Che  graph,  Che  sCate 
s^  represenCed  by  N^,  occurs  at  the  instant  n. 

For  a  proof  of  this  property  and  its  consequences  the  reader  is 
referred  to  references  18 and  19.  Using  the  above  property  one  may  obtain 
expressions  in  closed  form,  in  terms  of  the  individual  transition  probabilities, 
for  Che  various  probabilities  and  expected  values  values  introduced  in  section  O. 


2.  Permanent  Errors  (Type  II) 

In  Che  case  of  a  permanent  failure  of  a  subnet,  the  ability  of  a  self- 
correcting  net  to  successfully  repair  the  failure  will  depend  upon  che  nunber 
of  alternate  subnets  that  have  been  provided  to  the  net,  a  priori.  In  this 
case,  the  calculation  of  the  probability  of  occurrence  of  a  permanent  failure 
in  the  total  net  is  straightforward  and  no  special  mathematical  techniques 
are  called  for. 

In  the  next  section  we  shall  introduce  three  self-correcting  nets  and 
calculate  the  various  parameters  associated  with  them.  These  examples  are 
intended  primarily  f.o  illustrate  the  problems  involved  in  the  construction 
of  such  nets.  Also, an  understanding  may  be  obtained  of  the  orders  of 
magnitude  of  the  different  quantities  and  how  they  may  be  changed  with  changes 
in  individual  output  transition  probabilities. 
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3. 


Description  of  Some 


Typical  Self-Correcting  Wets 

The  outputs  yn  and  of  the  net,^,  in  Figure  12  are  given  by  the 
equations: 

■  f»  ®  •  Vi  +  fD®  •  Vl  <■'» 

*»  "  £d  ®  '  Vi  +  fD®  '  Vi  •  <I4> 

The  boxes  Fj  and  Gj  of  the  net  correspond  to  the  F  and  G  boxes  of  Figure 

10.  Inside  Fj  and  Gj  the  nets  Dp  D^  and  D^  denote  the  D-nets  that 
calculate  the  desired  function,  fp(X) .  These  four  nets  correspond  to  the 
four  occurrences  of  the  function  f^  (2)  in  equations  (13)  and  (14).  Indi¬ 
cating  this  correspondence  more  explicitly,  equations  (13)  and  (14)  suiy  be 
rewritten  as: 

,®  •  Vi  +  “ii  ®  ’  Vi  <36> 

*  fD,®  •  Vi  +  V®  •  Vi  <37> 

3  4 

The  gate,  marked  C  in  Figure  12  is  a  complementing  gate.  The  markings 
on  the  other  gates  are  self-explanatory.  In  this  net,  a  typical  output 
state  will  be  a  valuation  of  the  pair  (y,z).The  net  has,  in  all,  only  four 


states,  viz: 

(0, 

.0), 

(0,1), 

(1,0)  and  (1,1).  The  sets  So, 

ST(y)  and 

Sj(y)  for  y 

-  0, 

1,  are: 

S  - 

o 

{(0,1),  (1,1)). 

ST 

(0)  - 

V»-So  * 

SI 

(0)  - 

(0,0) 

and 

SI 

(1)  - 

(1,0). 

In  this  case 

the 

terminating  condition  has  been  chosen  as 

*n+1 

•  1  -  7„  ■  fD® 

and,  therefore,  the  terminating  state  itself  is  not  used  to  distinguish  between 
a  proper  and  an  erroneous  computation.  When  y  *  0,  a  computation  will  be 

*  In  this  case  the  condition  1  of  Definition  1  has  not  been  used. 
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proper  only  if  the  state  Sj(0)  ■  (0,0)  occurs  as  the  last  but  one  state  of 
the  computation.  So  also,  when  y  ■  1  the  state  S^(1)  ■  (1,0)  should  occur 
as  the  last  but  one  state,  for  a  proper  computation.  An  erroneous  compu¬ 
tation  will  occur  only  if  ■  1  and  yR  ^  f^(X) )  for  some  n  -  1,  2,  ...  . 

The  probability  of  occurrence  of  this  event  will  give  us  the  first  measure 
of  reliability  of  the  net,  ^ . 

Let* 


p'(Dj)  -a,  j  -  1,  2,  3,  4 


(39) 


be  the  first  measure  of  unreliability  for  the  nets  ,  j  ■  1 ,  2,  3,  4  in 
Figure  12.  So  also,  let 

p*(&)  -  p'(OR)  -  p'(C)  -  p  (40) 


be  the  first  measure  of  unreliability  of  the  other  gates  in  Figurel2,  that 
belong  to  the  control  part  of  the  self -correcting  net.  One  can  show  that 
the  first  measure  of  unreliability  of  the  net  Fj  will  be: 

p'(F,)  <  (a  +  30)  -  e,  say.  (41) 

For  the  purpose  of  our  calculations  we  shall  choose  the  upper  bound  as 
equal  to  p'(Fj). 

The  net  Gj  may  cause  two  types  of  errors,  type  A  and  type  B: 

Type  A:  is  the  event  ** 


{zn+1  “  °lyn  *  V*>J  • 


This  causes  a  recalculation  even  though  yn  »  fp(X),  thereby  increasing  the 
length  of  a  computation.  This  type  of  error  does  not  directly  cause  an 
erroneous  computation.  Let  us  denote  the  probability  of  this  event  by  e.. 

_  A 

j, 

p'(D  )  «  Max  p  where  p  is  defined  by; 

1  i  1  1 


V*i> 


W  with  probability  p^ 
fj^(X^)  with  probability  (1-pt) 


for  the  input,  X, . 


**  To  be  read  as  zn+J  ■  0  given  that  yn  ■  fp(X). 
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Type  B:  is  the  event 


!zn+J  *  1  I  yn  +  fD(*)]' 

This  error  causes  the  computation  to  be  erroneous.  Let,  e  ,  be  the  proba- 
bility  of  this  event. 

For  the  net  Gj  the  probabilities  of  occurrence  of  these  two  events 
can  be  shown  to  be: 

plVn  *  0  I  yn  "  fD(*>}  *  P{zn+1  •  1  I  yn  V  *„«>)  <  <42) 

We  shall  choose  in  our  ensuing  calculations 

p'<*,)  -  P'(G,)  «  e,  (43) 

for  the  net  ,  of  Figure  12. 

The  net ?2  in  Figure  13  is  a  modified  form  of  the  net  in  Figure  12. 

In  3"2,  greater  redundancy  has  been  introduced  in  the  calculation  of 


zn+1  “  Zn+1 


(44) 


where 


‘Li  ■  fD  \  ®  '  K  <«> 

3  4 

4  '  £D  <»•  yn  +  'fD,®  '  5„  «*> 

4  3 


In  this  scheme,  for  a  given  value  of  yn  both  D3  and  will  be  used 

to  calculate  s^,  •  If  one  of  the  quantities  z*+J,  i  -  1,  2,  indicate  that 

yn  is  erroneous  (i.e., z*  •  0  for  i  ■  I  or  2)  the  computation  will  be 

1  2 

continued.  A  termination  will  occur  if  and  only  if  both  z  ..  and  z  ,  ■  1  . 

iH-l  n-H 

For  this  net, 

P'(F2)  “  e,  (47) 

and  -  0  |yn  •  fD(£)}  <  e,(1  -  2  p)(2  *  «,)  +  P  -  eA  say,  (48) 

P(zn+1  "  1  I  yn  “  fD(*)}  <  £f(1  -  2  P>  +  P  -  eB  say,  (49) 
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Fig,  13.  The  net,  2F2 !  a  »odifi«d  form  of  Fj^ 

using  greater  redundancy  in  calculating  z  . 
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where  €j  is  defined  in  equation  41 .  One  may  notice  that  «  e^.  Thus, 
in  this  case,  calculations  in  a  computation  are  likely  to  be  more  often 
continued  than  be  terminated  erroneously. 

In  Figures  12  and  13  the  feedback  input,  z,  to  the  F-nets,  actually 
selects  an  output  from  one  of  the  two  D  nets,  Dj  and  Dj.  This  selection 
process  involves  the  use  of  additional  gates  that  reduce  the  first  measure 
of  reliability  of  the  output*,  y,  from  a  at  the  output  of  the  D-nets  to 
nearly  (a  +  3£)  at  the  output  of  the  delay  unit  (reference  to  equations  (39) 
to  (41)).  In  any  scheme  involving  the  use  of  alternate  nets  the  selection 
process  of  the  outputs  cannot  be  avoided.  However,  part  of  the  selection 
may  be  done  at  the  input  end  of  the  D-nets,  as  shown  in  Figure  14a.  In  this 
figure  the  input,  z,  is  used  to  activate  at  a  time  only  one  of  the  two  nets: 
Dj  when  z  »  1  and  Dj  when  z  »  0.  When  a  net  is  not  activated  its  output 
ray  be  assuned  to  be  zero**.  Thus  the  output  of  the  'OR'  gate  in  Figure 
14  a  will  be  the  same  as  the  output  of  the  net  that  had  been  activated.  Or 
else,  the  scheme  shown  in  Figure  ]Ab  may  be  used,  where  a  switch  has  been 
employed  to  select  the  appropriate  output.  If  the  probability  of  both  Dj 
and  D2  being  unactivated  is  7  and  the  probability  of  the  switches  (or  the 

(a)  (b) 


Fig.  14  Illustrating  the  use  of  z  as  an 
actuating  signal. 


*  Note  that  the  reliability  of  a  net  is  considered  to  be  the  same  as  the 
reliability  of  its  output. 

**  In  a  typical  net,  in  practice,  the  activation  of  a  net  may  be  caused  by 
providing  the  power  supply  necessary  to  operate  a  net.  When  no  power 
is  supplied  to  a  net,  its  output  has  to  be  zero  in  the  steady  state. 
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OR  gate)  making  a  wrong  selection  is  3,  then  the  first  measure  of  reliability 
of  the  output  y  will  less  than  (a  +  3  +  y).  If  more  than  one  feedback  line 
were  available  then  more  than  one  D  net  may  be  actuated  in  each  calculation. 

In  such  a  case,  the  output  selection  may  be  more  profitably  done  by  a  ma-  ■ 
jority  gate  instead  of  an  OR  gate  or  a  switch.  The  use  of  a  majority  gate 
for  output  selection  will  result  in  a  considerable  improvement  in  reliability. 
Table  III  is  a  sumnary  of  the  different  probabilities  of  F  and  G  units  in 
Figures  12  and  13. 


Table  III:  The  Probabilities  Associated  with  the  Subnets 

of  Figures  12  and  13. 


;  Subnet 

"  ' 

Probability 

Di 

p'(D  )  ■  a  for  all  i 

Any  control 

gate.&,  OR, 

p'(&)  -  3 

C,  etc. 

| 

Fj  of  Fig.  12 

p'(Fj)  5  e,  -  a  +  33 

Gj  of  Fig.  12 

p'  (Gj)  5  e1  -  eA  -  ep  -  a  +  33 

F2  of  Fig.  13 

p'(F2)  5  e,  -  a  +  33 

G2  of  Fig.  13 

P{znf1  “  °lyn  "  fD(*>}  '  CA”  e,  (1-23X2-6,)  +3 

p(Vi  '  '  1  ♦  £i>®) 5  +  e 

4.  Analysis  of  the  Sets  Described  in  Section  3. 


The  Markov 
will  be  as  shown 
Figure  15  are  for 


chain  diagram  for  the  nets  ^  and  ^  of  Figures  12  and  13 
in  Figure  15.  The  transition  probabilities,  shown  in 
the  case:  fp(X)  ■  1  and  initial  state,  (0,1). 
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Fig.  15  Markov  chain  diagram  for  che  nets  7  ^  and 

for  the  case:  Initial  state  (0,1)  and  f^ (X^)  -  1. 

The  probabilities  €,,  €A,  Eg  in  Figure  15  are  given  in  Table  III  and  q,  &  are 
given  by: 

tj  -  1  -  (e,  +  cB  +  e,  •  eB)  (53) 

and 

6  »  1  -  (€j  +  €A  +  €,  •  cA)  (54) 

The  probabilities  q  and  6  indicate  the  correct  transitions,  as  given  in 
Table  I,  for  the  case,  fQ(X)  ■  1  (rows  1,  3,  5,  and  7  of  the  table).  The 
freight  6j  Indicates  an  erroneous  transition  —  not  as  required  by  Table  I 
—  where  the  output,  y,  in  the  'next  state'  is  in  error.  The  weights  eA 
and  6g  indicate  that  the  outputs  z  in  the  'next  states'  are  in  error,  the 
errors  being  of  type  A  in  one  case,  and  type  B  in  the  other.  The  weights 
€j  •  cA  and  Ej  •  €g  indicate  that  both  y  and  z  are  in  error  in  the  'next 
states'  of  the  transitions.  As  the  state,  (1,1),  in  Figure  15  will  always 
be  a  terminating  state,  no  transitions  are  shown  emanating  from  the  state. 

Using  Table  I  and  the  error-probabilities  shown  in  Table  III  the 

tc 

See  Table  III  for  definition  of  cA  and  Cg. 


] 

1 

I 

I 

1 

:i 

-  r 
.1 


1 

J 

1 

I 

I 


I 
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Markov  chain  diagram  for  any  initial  state  and  output  value  may  be  similarly 
drawn.  In  Figure  15,  for  example,  if  the  states  (1,0),  (0,0)  are  transposed 
and,  so  also,  (1,1),  (0,1)  are  transposed  the  resulting  diagram  will  repre¬ 
sent  the  case:  fp(X)  “  0  and  initial  state,  (1,1).  One  may  verify  that 
the  basic  structure  of  the  diagram  for  all  possible  initial  and  output  states 
remains  the  same.  Thus,  the  probabilities  of  proper,  and  erroneous  compu¬ 
tations  calculated  for  Figuiel5will  remain  unchanged  for  other  cases  also. 

Using  the  z-transform  techniques  and  the  analogy  of  signal  flow  graphs, 
one  may  obtain  the  following  expression  for  the  different  statistical  quanti¬ 
ties  associated  with  the  Markov  chain  diagram  of  Figure  15.  Let, 

00 

^  Pp(i)  •  z_i  -  Pp(z)  (55) 

1-2 

and 

OO 

^  Pe(i)  •  z_i  -  Pe(z)  .  (56) 

1-2 


Assuming  that  the  length  of  each  computation  is  >  2,  we  get  that. 


and 

P-0)  “  Pn 

P  P 

P  (1)  -  P 
e  '  e 

(57) 

(58) 

For  the  Figure  15, 

*,(')  " 

T|  z"1  (6  +  € y  ) 

(1-€j  z*1)(1-eA  z1)  •  n  €,  €a  z'2 

(59) 

£e<o  - 

e1  °'eA  2’’<,-T>»  eB°  +  €1> 

(1-e,  z  1 ) ( 1  - eA  z  l)  -  £j  eA  t)  z-2 

(60) 

E<y  - 

2  -  £i  -  €a 
,_€i  -  £a  +  €i  €a 

(61) 
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(62) 


V(ip) 


£1  +  e2  -  £1  eA  (1-^(4~£1  ~  €A> 
(,'e1  *  eA  +  £1  £A  (,'^»2 


1_£A  O'^X2*^ 

E(ie)  -  1  +  (j-eB  (1-Ti))(1-e1  -  eA  +  e,  €A<1  ~n>) 


(63) 


v(*e) 


6,  +  eA  (Sg-4)  -  86a2  (1  -T|)2 
(1-eB  (1-T|))2  (1-6j  -  eA  +  e,  eA0-n))2 


(64) 


The  values  of  Pp(4)  have  been  plotted  in  Figure  16  for  various  values 
of  the  parameters  a  and  k  where, 


k  *  a/p. 


(65) 


a  and  p  are  defined  in  Table  III.  In  a  sense,  'k'  is  a  measure  of  the 
relative  complexity  of  the  D-nets  in  Figures  12  and  13  as  compared  to  the 
individual  control  gates  —  AND,  OR  or  C.  The  larger  the  value  of  k  is,  one 
may  say  that  the  greater  would  be  the  complexity  of  the  D-nets  as  compared 
to  the  individual  AND,  OR  or  C  gates.  One  may  notice  that  in  the  net, 
the  computations  tend  to  be  longer. 

The  values  of  Pp,  Pg,  E(£p)  and  E(ig)  are  tabulated  in  Table  IV.  One 

may  notice  that  for  k  *  1,  there  is  a  loss  in  reliability  for  a  *  .05.  When 

k  ■  10  the  gain  in  reliability  is  ignificant.  Also,  in  all  cases  the  net, 

gives  greater  improvement  in  reliability  at  the  cost  of  increased 

average  delay  time  for  computation. 

It  may  also  be  seen  that  in  all  cases  E(ig)  >  E(^p).  This  does  not 
satisfy  the  condition  given  in  inequality  (30).  It  remains  to  be  seen 
whether  it  is  possible  to  satisfy  (30)  through  the  use  of  this  scheme  of 
self-correction,  and  if  it  is,  what  the  requirements  on  the  feedback  net 
will  be. 
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Table  IV:  Calculated  Values  of  P  ,  P#,  E(/p)  and  E (Jtj 


■ 

p 

P 

P 

e 

E(fp) 

E(ie) 

a 

1 

2 

1 

2 

■1 

2 

m 

2 

0.05 

1 

.91095 

.96743 

.08905 

.03257 

2.5906 

3.0050 

2.4942 

2.6846 

10 

.9945 

.99952 

.0055 

.00048 

2.148 

2.236 

2.139 

2.209 

100 

.99967 

.99996 

.00033 

.00004 

2.114 

2.117 

2.108 

2.1613 

0.01 

1 

.99981 

.999949 

.00019 

.000051 

2.086 

2.144 

2.083 

2.134 

10 

.999983 

.999986 

.17x10*4 

.14x1 0 

2.026 

2.041 

2.026 

2.040 

100 

.999989 

.999999 

. 1 1x1 O-4 

.10x10"5 

2.021 

2.031 

2.020 

2.031 

0.001 

1 

.999984 

.999995 

. 16x10~4 

.41x1 0*^ 

2.008 

2.013 

2.008 

2.013 

10 

.999998 

.9999999 

.17x10*5 

. 1 3x1 0~6 

2.002 

2.004 

2.002 

2.004 

100 

.9999989 

.99999999 

.11x1 0-5 

. 1 0x 10* 7 

2.002 

2.003 

2.002 

2.003 

The  nets  and  °f  Figures  12  and  Irrespectively,  do  not  provide 
much  protection  against  permanent  failures.  No  redundant  gates  have  been 
provided  In  the  control  network.  As  the  control  gates  are  simple  AND,  OR 
gates,  one  may  Introduce  redundant  gates  through  the  ' triangulation '  scheme 
of  Levy  I*"  and  Amarel.^  Let  us  assume  that  such  a  scheme  has  been  used  and 
the  probability  of  permanent  failure  of  the  control  part  of  the  F  and  G 
nets  Is,  p"(c).  In  and  ^ • 

Let  us  now  consider  the  failure  of  one  of  the  D-nets.  In  the  F-nets 
of  both  Figures  12  and  11  failure  of  either  Dj  or  Dj  will  not  cause  a  failure 
of  the  total  F-net.  The  self-correcting  feature. —  namely,  the  substitution 
of  an  alternate  net  in  place  of  an  erroneous  one  —  will  assure  us  of  the 
correct  output  in  each  computation.  Thus  the  F-nets  of  ^  and  ^  will  fail 
only  if  both  Dj  and  Dg  fall,  or  the  control  part  of  the  net  fails.  There¬ 
fore/' 


p"  is  the  second  measure  of  unreliability. 
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T 

4 


5  6  7 

Length  of  propi 


2  3  4 

r  computation 


Fig.  16  Plots  of  probability  of  correct  computation  against 
lengths  of  the  computations  for  various  values  of 
a  and  k,  for  the  nets  5^  and  J 


(66) 


P"  (F^  -  {p"(D))2  +  p"(C),  i  -  1,  2 

for  the  nets  ^  and 

In  the  case  of  the  G-nets  of  ^  and  failure  of  either  or 
will  cause  a  failure  of  self-correcting  property.  Further,  if  z  gets  stuck 
at  0,  no  computation  will  ever  terminate.  So, 

P"  (G^  -  p"(D)  +  p"(C),  i  -  1,  2  (67) 

However,  when  fails  the  nets  cf^  and  will  operate  properly  for 
the  output  y  ■  1,  and  also,  when  fails,  proper  operation  will  be  available 
for  y  *  0.  Unless  z  gets  stuck  at  0  the  overall  nets  sfj  and  ^2  may  still  be 
used,  even  though  the  self-correcting  feature  will  be  absent  for  some  outputs. 
The  probability  of  z  getting  stuck  at  0  will  be  less  than  p"(D)  +  p"(C).  Thus, 
some  improvement  will  be  obtained  in  the  overall  reliability  of  the  nets  ^ 
and  sf2*  *n  general,  one  may  choose  a  weighted  sian  of  p"(F^)  and  p"(G^)  for 

p"(5k>,  1  -  >>  2- 

p"(  dfp  -  cp^  p"(F1)  +  c  •  p-CCp  (68) 

for  i  ■  1,  2,  where  c_  ■  1  and  c_  <1,  are  constant  coefficients. 

Fi  Gi 

The  above  discussion  was  intended  only  to  illustrate  the  techniques 
of  calculation  and  relative  values  of  the  various  parameters  that  may  be 
associated  with  a  self-correcting  net. 

One  may  intuitively  see  that  there  exists  a  great  variety  of  possi¬ 
bilities  in  which  such  nets  may  be  constructed  when  the  number  of  feedback 
lines  is  large.  A  greater  understanding  of  the  subject  will  have  to  be 
developed  before  a  definite  conclusion  may  be  made  about  the  practical 
applicability  of  the  scheme  of  self-correction,  introduced  in  this  report. 
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